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PREFACE 


The purpose of this volume is to make available the lesults of work that 
has been done during the past decade in the Psyehometnc Laboratory at 
the University of Chicago The material is piesented m the form of a text 
for students Essentially all the material of the first edition, The Vectors of 
Mind, has been retained heie, and much has been added both in multiple* 
factor theory and in computational method Detailed computational office 
routine has not been featured because it will be covered in a workbook on 
multiple-factor analysis. 

The Vectors of Mind has been out of punt twice, and the cun cut loprint- 
ing is now exhausted When the writing of the present volume was started, 
it was my intention to revise The Vectors of Mind, re-writing certain sec¬ 
tions and making additions of new mateiial This plan was altered by 
adopting a more complete expository style for the piesent text The entiie 
book has been re-written, and hence it has been given a new title The Vec¬ 
tors of Mind will be referred to in this text as the “fiist edition " 

It has not been my intention to produce a bibhographically complete 
treatise on multiple-factor analysis That would be entuely beyond the 
scope of the piesent text Only those icfcicnees have been given that were 
directly in line with the subject ol discussion, but I fear that even with this 
limitation I have not done justice to all the authors whose woilc should 
have been summarized in this text Most of these refeienccs me readily 
available in the journal Psychometrika and in the bibliogi apluos of its many 
papers on factorial theory. The subject is stdl advancing rapidly, so that a 
theoretically complete tieatise would become obsolete in a few yeais When 
the time comes for such a volume, it should be written by a mathematician 
or by an author who is competent to appraise the contributions of mathe¬ 
maticians and physicists m this field. 

The brevity of The Vectors of Mind was piobably responsible for much 
of the controversy about multiple-factoi analysis As far os I am aware, all 
the theorems in the first edition are as valid today as they were ton years 
ago The theorems were stated, and, in general, they were pioved; but I 
left them without the support of exposition as to their implications and in¬ 
terpretation in exploratory scientific woilc. Several concepts have been the 
subject of controversy It has been a mystery to mo why tiie fundamentally 
simple notion which I called "simple structure’' has been twisted mound 
by some critics into the most inconceivable nonsense. Tlieio has been 
misunderstanding about the eommunality concept by which multiple- 
factor analysis is limited to the common factors- -just as it is in Spearman's 
single-factoi case Some students of this subject change their set in going to 
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the multiple-factor case by insisting that they must then include the total 
test space There has also been confusion about the oblique reference frame 
which represents parameters or factors that are correlated m the general 
population as well as m the experimental population 

When multiple-factor analysis was introduced in 1931,* it constituted a 
shift of emphasis Papers being published at that time were mainly con¬ 
cerned with the existence of Spearman’s central intellective factor g, the 
tetrad difference criterion for the general factor g, probable error formulae 
for the tetrad difference, and group factors which were regarded as the dis¬ 
turbers of the general factor and which were nearly always regarded as 
residuals that were secondary to the general factor. There is nothing new 
in the recognition that more than one factor must be postulated to account 
for the observed test correlations Instead of asking whether the central 
intellective factor g can be demonstrated in any given set of correlations, 
with or without disturbing group factors, we ask how many factors are in¬ 
dicated by the correlations without restriction as to whether they are gen¬ 
eral or group factors Instead of extracting, first, the postulated general 
factor and then investigating the residuals to determine whether any dis¬ 
turbing gi oup factors must also be admitted, we start with an observation 
equation m n terms which represent as many factors as may be required 
by the correlations It becomes, then, a question of fact as to whether one 
or more of them are general factors and whether one or more of them are 
positive or bipolar 

Our matrix formulation of the problem led to the fundamental factor 
theorem FF' ~ R, where F is a factoi matrix and R is the correlation ma¬ 
trix The matrix formulation of factor analysis seems to have been generally 
accepted, and it has largely replaced the previous methods in factor 
analysis. 


In 1931 I decided to investigate the relation between multiple-factor 
analysis and Spearman’s tetrad differences When I wrote the tetrad equa¬ 
tion to begin this inquiry, I discovered that the tetrad was merely the ex¬ 
pansion of a second-order minor, and the relation was then obvious One 
might speculate as to whether multiple-factor analysis would have de¬ 
veloped earlier if this interpretation had been stated earlier If the second- 
order minors must vanish in order to established a single common factor 
then must the third-order minors vanish m order to establish two common 
factors, and so on? To have put the matter in this way would have led to 
the matrix formulation of the problem much earlier, as well as to the im¬ 
mediate development of multiple-factor analysis Instead of dealing with 
the proportional columns and rows of a hierarchy and the vanishing tet¬ 
rads, we now deal with the same relations m terms of the properties of unit 
rank, namely, proportional columns and rows and vanishing second-order 
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minors This formulation generalizes to the piopcrties of higher lank. As 
far as I am aware, this foi mulation of the factor problem has been generally 
accepted 

The test conelations define a configuration of test vectors without a ref¬ 
erence frame Since the factois are repiesentod by the axes of this flame, if 
is necessary to locate a reference flame somehow in the test eonfigui ation 
Here we have to recognize a distinction between an aibitrary orthogonal 
reference fiame that is used foi computing purposes and the refeieuee 
fiame that should be used for scientific interrelation of the fat bus The 
arbitrary orthogonal frame is defined by the method of factoimg that, the 
computer happens to use In order to locate a lcfeienee fiame foi mterpio- 
tation, I used, at first, a statistical cuterion which led to the puneipal a\es 
of the eonfigui ation, but I soon discarded this solution m favoi of ti pnn- 
ciple that I called "simple structure ” This principle of simple stmetuie 
has been the cause of much controvcisy, and it is not yet accepted by all 
students of factorial theoiy The fact that it veiy frequently gives a set of 
factors or parameters that can be mtcipictcd as meaningful in scientific 
context would seem to be an aigumcnt in its favor, but even thill insult 
has been the object of contioversy 1 have found that the best way to con¬ 
vince a skeptic is to let him woik with the lotational methods on a piob- 
lem in which he is mtciested. As llio lefeicnco frame begins to eineige in 
the rotational woik with successively eleaiei mteipretatiou, lu‘ eommees 
himself of the powei of the method 

In locating a meaningful lefei cnee fiame* foi a sol of euneluted vat tables, 
one may voluntarily impose the lostnction that the lefoionco frame shall 
be orthogonal, or one may allow it to become oblique uceoidmg to the test 
configuration If we impose the resluotion that the reference fiame shall 
be orthogonal, then wc aie imposing the condition that the factois 01 
parameters shall be uncorrelated in the expeiirnental population or m tin* 
geneial population It is my own conviction that this restuetion should 
not be imposed if we are looking for meaningful paiameters If. seems just 
as unnecessaiy to lequire that mental tiaits shall be uncoil elated in the 
genei al population as to require that height and weight be uneoi i ejul ed in 
the general population This admission of oblique axes has also been de¬ 
bated and is not yet generally accepted by students of faetoi theoiy 

The conelations among pnmaiy factois must be expected to vary finm 
one expeiirnental population to another, depending oil selective comilt ions 
This does not mean that the physical nature of each factor 01 puiuiuuler 
has alteied Just as the coirelation between height and weight vanes fiom 
one group to another by selective conditions, so the conelations between 
mental tiaits may also vaiy from one gioup to anothei without change, m 
their meaning 

Second-oidei factors which aie determined from the correlated pnmimes 
have only lecently been introduced, so that it is not yet known how fruitful 
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they will be It seems likely that a second-order general factor, determined 
from correlated pnmaries, may turn out to be Speannan’s general intellec¬ 
tive factor g, Spearman’s early admonition that the tests in each tetrad 
must not be very similar may be the equivalent of our present observation 
that a set of foui tests can be selected so as to involve only one primary fac¬ 
tor for each test. Such a set of four tests will have only the second-order 
general factor m common; hence the correlations will be of unit rank, and 
the second-order minor s must then vanish When two similai tests are in¬ 
cluded in a tetrad, they are likely to involve the same primary as well as 
the second-order general factor, and the unit rank is then disturbed 
In the scientific interpretation of factorial results it is necessary to dif¬ 
ferentiate between what we have called “configurational invariance” and 
the numerical mvanance of correlations and saturation coefficients In 
geneial, configurational invariance is more significant, since it transcends 
most selective conditions for the groups of experimental subjects Those 
authors who insist on avoiding geometrical interpretations in factor anal¬ 
ysis will almost certainly have some difficulty in dealing with configura¬ 
tional invariance and its important implications The same results can 
usually be stated m analytical form, but it is likely that the geometrical 
ideas will precede the analytical foims 
Since the publication of The Vectors of Mind, theie have been several 
texts on factorial theory The text by Holzingei and Harman presents fac¬ 
tor analysis as a statistical method * This is a worth-while objective, but 
the mathematical statisticians have not yet accepted factor analysis as a 
part of statistical logic Holzmger describes several factoring methods, in¬ 
cluding his own bi-factor method The simple-structure concept is disposed 
of in a footnote, but the rotational problem is presented m variant forms 
of factorial resolution It is unfortunate that the term “structure” has been 
defined in different ways Holzmger accepts the matrix formulation of the 
multiple-factor problem and the interpretation of Spearman’s single-factor 
hypothesis as a special case of unit rank The oblique axes are descubed 
as alternative solutions 


Truman L Kelley’s monograph, Essential Trails of Mental Life f de- 
sciibes an ingenious method of arimng at the principal-axes solution di¬ 
rectly from the eon elation matrix Kelley does not accept the communah- 
tms, or the simple-structure concept, or rotation from the piincipal axes 
tor the purpose of isolating simpler and more meaningful parameters. He 
attempts to interpret directly the principal axes of each test battery Kelley 
insists that factor analysis must be made with unit diagonals to represent 
e otal variance rathei than the common-factor vanance His earlier 
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monograph, Crossroads in the Mind of Man,* should also be aokn.owle.dged 
as a significant contnbution In that publication he attempted to break 
away from the dominant single geneial factor to the multiple-factor piob- 
lem, but he did not then introduce the generalized factor problem 

Burtf accepts the communality concept, the matiix formulation of fac¬ 
tor analysis, and the mlerpi etation of Spearman’s single-factor hypothesis 
as a special case of unit rank, but he does not accept the necessity for ro¬ 
tating the reference fiame for pui poses of interpretation Neither does lie 
accept the use of oblique reference axes The configurational interpreta¬ 
tions are evidently distasteful to Buit, for he does not have a single dia¬ 
gram m his text Perhaps this is indicative of individual differences in 
imageiy types which lead to differences in methods and interpretation 
among scientists 

The most piovocativc of the lecent texts in factor theory is that of (lod- 
fiey H Thomson % Even when he diffeis from our intcipretntions of the 
factor problem [ find his discussion stimulating for our own work Our 
chapter on selective conditions and sections of the chapter on unit lank 
resulted directly from the stimulus of his provocative papois. lie docs not 
seem to be convinced about the necessity for rotating the inference fiame, 
although he is piobably open-minded about the question lie is not opti¬ 
mistic about the inteipretation of ptimary factors as fundamental parame¬ 
ters in the description of mentality 

While there has been considciablo contiovcrsy about multiplc-lnctoj 
analysis, it is probably tiue that the large majority of factorial studies in 
iecent yeais have accepted the malax formulation of the problem, the 
interpretation ot Speaiman's single-tactor theorems as a special ease of 
unit rank, the cominunalities, the simple-structure concept, the necessity 
for rotating the refeicnee frame for scientific interpretation, the desira¬ 
bility of interpreting the pnmaiy factors as meaningful parameters, the 
use of oblique leference axes, and the principles of configurational in- 
vanance 

No attempt has been made hero to review the numerous applications of 
factor analysis to psychological and other pioblcms Dael Woltie’s mono¬ 
graph, 1<actor Analysis to 1940 ,§ was a good suimnury of thcoiy and of 
applications, and such reviews should be made periodically eithei by new 
monographs like Wolfle’s or by periodic sununaiies in the Psychological 
Bulletin. I'or several reasons in the present text I have used very few psy¬ 
chological test examples When psychologists start to work with psyolm- 

*Stanfoul, Calif Stanford TJmvaimty Press, 1028. 

t Cyul Buit, The Faclois of Che Mind (Now Yotk Macmillan Co , 1041), 

t The Factorial Analysis of Human Ability (New York’ Houghton Mifflin Co., 1080) 

§ "Psychometnr. Monogiaphs,” No 3 (Chicago- University of Chicago Press, 1040). 
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logical test conelations to find primary mental abilities, they sometimes 
jump the track and fall into a discussion about faculty psychology The 
logic of the subject can be presented perhaps more clearly by using physical 
objects That is the reason we have used populations of rectangular boxes, 
populations of cylmdeis, and a population of trapezoids Here the student’s 
attention is directed to the interpretation of factors as parameters in the 
usual scientific sense In this way we attempt to make the relations of the 
paiameters in factonal repiesentation as vivid as possible 

One of the limitations of factor analysis m its present stage of develop¬ 
ment is the absence of sampling-error formulae It is peihaps natural that 
factoiial logic should be developed before the sampling-error formulae It 
is only lecently that the influence of selection of tests and ot experimental 
subjects on factorial results have been investigated These effects must be 
distinguished as to the conditions that pioduce stable numeucal values 
and those in which configurational invariance is of pumary interest The 
relations of second-order factors to factorial results have also only recently 
been investigated At present we are limited largely to the empirically ob¬ 
served stability of factorial findings. The availability of sampling-error 
formulae for the principal-axes solution for a given correlation matrix is 
good as far as it goes, but this is a tufle compared with the sampling- 
error pioblems m factoi analysis 

Factor analysis has not been generally accepted by mathematical statis¬ 
ticians The present conceptual formulations of statistical theory do not 
seem to lend themselves readily to the logical problems of factor analysis 
One might resolve this difficulty in one of two ways One might insist that 
multiple-factor analysis bo re-written within the lestnctions of current 
statistical theory, or one might ask the statisticians to adapt and expand 
their formulations to covei the logical problems of factor analysis Peihaps 
both ways will be tried 


When the mathematical statisticians eventually decide to assist the stu¬ 
dents of multiple-factoi analysis, it will be helpful if certain fundamental 
considerations are lecogmzed at the outset Some of these can be listed 
eie First, let us recognize that when a scientist is trying to invent a theory 
to cover the phenomena of a particular domain, a simple description of its 
underlying older, he is certainly not concerned with sampling distiibutions 
furthermore he must lestnet himself to some aspect of the phenomena m 
the domain He cannot hope to cover the total variation, for no scientific 
, rmulatlon ever covers the total variance m any domain When the atatis- 
" nS hW ^ t0 th0 multi Ple-factor problem, he must deal 

main ?I “““P “ d ltS 6ffeCt ° n the rank of the con elation 
" ? 16 d f les J lls Problem, ho not only admits his inadequacy but 
also his failure to understand the problem If statistical theory is nofnow 
adequate to cope with the commonalities, then the statistician has the 
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challenge to adapt lus foi mutations to the limitations that ai c inheient in a 
very large class of scientific pioblems He need not deal with the column- 
nalities as such because he may discovei some equivalent insolation, but he 
will not solve the problem by laughing it out of comb. 

Another consideiation with which the statistician is already familial is 
the distoition intiodueed by the assumption of linearity in the exploratory 
study of a new domain Tieatises in the moio advanced sciences abound 
with non-linear relations Linear iclations aie the exception. Them is no 
good reason to believe that iclations in the youngci sciences ate funda¬ 
mentally simplci When scientific mquny enters upon a new domain with 
hazy ideas about the undeilying oidei or, perhaps, with no ideas at all, the 
scientist is as naive as we would be if we should make an cxpei unoutul in- 
quuy about the 1 elation between the sizes and the weights of boxes with¬ 
out knowing their contents tie would measure and weigh the lust one hun¬ 
dred boxes and detenmne the coi 1 elations and the regression equations, 
and then he might worry about the standard cuors of Ins eoedieients 
Something would be gained by doing that, because it would afloct the de¬ 
gree of assuianee with which he could say something about the iclations 
that he had found If the scientist takes his numeneal coefficients veiv 
senously at the exploratoiy stage, lie may be lacking in a clean able sense of 
humor about the ciudencss of all Ins tools in spite of then polished appeal - 
ance Too much concern with numerical minutiae at that stage may lead 
him astray fiom the conceptual formulations that constitute his leal goal 
If factor analysis is one of Ins tools for the cxplm ation of a new domain, his 
principal concern will be with the. factor pattern, the* piesence or absence 
of a significant coefficient in each cell of the oblique factor matiix, lalhei 
than the numerical values of any of them This consideration may simplify 
the statistician’s problem in faetoi analysis 

So far, one of the principal handicaps of the statistician in dealing with 
multiple-factor analysis has been his failiuo to appieeiate the significance 
of the lotational pioblcm Tins is the problem of choosing the most, fuutful 
set of paiarneteis to desenbe the vauation in a domain These paiumoteis 


lepiesent scientific concepts They aie not mciely numeneal coefficients 
If these scientifically meaningful paiarneteis happen to be eon elated m 
the experimental sample, such as height and weight, (hut is no icason foi 
rejecting them The prioi considei ation is to find fuutful and meaningful 
parameteis Ihc question of whether the eioss pioduets vanish is a tnviul 
consideration of the eomputei’s convenience The .statistician must iiruloi- 
stand this pioblem of oblique reference axes and associated seieiililu con¬ 
cepts The statistician must not impose the arbitrary icst action I,lull, mu 
scientific concepts in the dcscnption of people must be uneon elated either 
m the expeiimental sample or in the general population 

In attempting to introduce sampling distubutions in factor analysis, it 
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will also be useful to recognize that when a simple structure is discovered m 
a new domain, it is rarely equally stable or clear in all the dimensions Some 
of the hyperplanes may be overdetermmed while others are even indeter¬ 
minate These hyperplanes are usually unknown at the start of the experi¬ 
ment, and no statistical jugglery can evade their different degrees of over¬ 
determination unless the jugglery merely obscures them 

One of the most useful conceptual formulations in statistical theory is 
that of the sample and the universe from which the sample is drawn A 
large part of statistical theory is built on this construct This often leads to 
a concern as to whether any given sample is representative of a postulated 
universe One might suppose in factor analysis that the experimental sam¬ 
ple should be representative of the general population, but such is not the 
ease On the contrary, if a certain group of traits is to be investigated as to 
their underlying structure, we proceed as in many other types of scientific 
investigation by selecting a highly biased sample, including even fi eaks m 
the domain concerned Another sample of freaks may be factorially studied 
without any question as to whether it is also an unbiased sample from the 
same universe Such studies aie much more likely to be revealing of the 
underlying nature of the domain than are carefully randomized samples 
from the general population And, further, there is no necessity that the fre¬ 
quency distribution of anything shall be Gaussian Let us also not foiget a 
simple principle that every scientist takes for granted, namely, that lie 
would rather measure something significant without any sampling distribu¬ 
tions than measure something trivial or irrelevant because its sampling 
distribution is known 

In a recent paper Hotelling deplores the fact that students of factor anal¬ 
ysis have not made use of the recent advances m mathematical statistics, 
and he dismisses the fundamental concepts of multiple-factor analysis be¬ 
cause they have not been formulated m terms of current statistical theory 
A fair appraisal of this situation would probably acknowledge that we 
should have succeeded better in getting the co-operation of the statisticians 
if recent advances in statistical theory had been recognized explicitly; but 
it would probably also acknowledge that, while statistical theory has made 
important advances in the last two decades, it has not advanced far enough 
to be immediately useful for factor analysis as a scientific method without 
still further advancement Instead of dismissing the basic concepts of 
multiple-factor analysis, we must reach common ground m recognizing 
that these concepts represent difficult problems in the logic of science that 
need to be studied rathei than ridiculed Let us hope that competent stu¬ 
dents will begin soon to investigate these challenging problems, so that the 
factorial methods can be made generally available m the sciences without 
the handicap of distracting controversy 
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A NOTE ON FUTURE WORK 

In presenting this text as a summary of our work on multiple-factor 
analysis, it may be appropriate to mention some possibilities for the future 
development of this subject One of the principal assumptions underlying 
factorial theory is that the scores are monotonic increasing or decreasing 
functions of the scores on the primary factors or parameters The funda¬ 
mental observation equation of factor analysis makes the further assump¬ 
tion that these functions can be expressed in linear form as a first approxi¬ 
mation It would be possible to start with a second-degree observation 
equation and to develop factorial methods on that basis Instead of devel¬ 
oping factorial theory more completely with observation equations of 
higher degree, it would probably bo more piofitablc to develop non-metric 
methods of factor analysis. An idea for such a development would be to 
determine the number of independent parameters of a score matrix by 
analyzing successive differences in rank order on the assumption that they 
are monotonic functions of a limited number of independent parameters. A 
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.core would then be regarded as merely an index of iank Older, and that is 
essentially what we are now doing The raw scores are transmuted into a 
lormal distribution of unit standard deviation, and these transmuted 
scores are used for the correlations Instead of dealing with the transmuted 
scores in this manner, one might deal with the rank orders directly or with 
some equivalent indices of rank order Analysis of successive differences in 
rank order, or their equivalents, should lead to the factor pattern, which 
would indicate the number of paiameteis involved m the variance of each 
test, That is the mam object of factor analysis The actual numerical values 
are of secondary importance in teasing out what we have called the under¬ 
lying order of a new domain The simple-structure concept is applicable 
as well to a non-metric form of factor pattern as to the more restricted 
linear metric form that we have been using The scientific interpretation 
of the factors could be made from a factor pattern without putting it in 
metnc form If the mathematical theory has already been developed, it 
would be helpful if we knew what to call it m asking mathematicians for 
assistance In putting the results to pi aetical use, the problem would return 
to a metric form, with standardization and norms which call for statistical 
methods of the conventional kind The mam pioblem for factor analysis 
is the more fundamental one of charting the alternative factor patterns in 
new fields, To develop this subject with more poweiful non-metric methods 
is future work that may leave our present efforts obsolete 

L L, Thurstone 

Chicago, Illinois 
March 1945 
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MATHEMATICAL INTRODUCTION 


The matrix theory which is used in the development of factor analysis is 
not generally available to students whose training m mathematics is limited 
to undergraduate courses m analytical geometry and in the calculus. This 
mathematical introduction reviews the elementary theory of matrices as 
well as the closely related theory of determinants. Summation that involves 
double subscript notation is included in this section, since it is used in factor 
theory and since it is unfamiliar to most students of statistics. In the geo¬ 
metrical interpretation of the factor lal matrix, only non-hornogeneous co¬ 
ordinates are used. For this reason, the introduction includes non-homo- 
geneous co-ordmates and omits homogeneous co-ordinates which are con¬ 
ventional. Orthogonal and oblique transformations have been illustrated 
geometrically No provocation has been found so fai in factor theory to in¬ 
troduce imaginaries and complex numbers, but the future development of 
factor analysis may call for them. This mathematical introduction is limited 
to the real case, and all theorems have been written with this restriction in 
mind. 

If this introduction is not self-sufficient, perhaps it may serve as a useful 
guide to the student of factor theory who seeks mathematical assistance on 
specified topics If a student has the intention of attaining some competence 
in factor theory and in related statistical work, there is no short cut for 
formal courses in the mathematics that is involved.* 


Matrices 


Matrices and determinants involve rectangular arrangements of num¬ 
bers Any rectangular arrangement of numbers is called a main t, lnespec- 
tive of what the numbers mean If the matrix has m rows and n columns, 
the matrix is said to be of m dei m X n In designating the order of a mat rix 


* The following references will bo found useful 

W F Osgood and W C Graustein, Plane and Solid Analytic Geometry (New V nrk 
Macmillan Co , 1929) 


L E Dickson, Modem Algebraic Theories (New York B II Sanborn, 1020), chap in 
Maxime B6eher, Introduction to Higher Algebra (New York Macmillan Co 1931) 
chaps i-vi, me ' ' ' 

FI W Turnbull and A. C Aitken, Canonical Matrices (London Blatkio A Sons 
1932), chap i 1 

V Snyder and C II Sisam, Analytical Geometry of Spate (New York- Henry Holt* 
Uo VJ 1914) 

A Adrian Albeit, Introduction to Algebraic Theones (Chicago- Umvermlv of 01,1, ago 

ITILSSj 


W L Ferrar, Algebra (Oxford Univeisity Picss, 1941) 

Arnold Diesden, Solid Analytical Geometry and Determinants (New York: John Wilev 

toons. lQ30'i J 


i 



2 MULTIPLE-FACTOR ANALYSIS 

it is customary to refer to rows first and columns second Thus a matrix of 
order pXg has p rows and q columns Tables la and lb show a matrix of 
order 3X4 and a matrix of order 3X3 A low is horizontal A column is 
vertical The general name for either a row or a column is an array Each 
of the small squares into which a matrix is divided is called a cell, and the 
number in each cell is called a cell entry or element 

Table la Table lb 

2 3 1 5 2 1 5 

1 6 0 9 4 8 3 

0 2 6 7 2 0 7 

In order to designate a particular element, it is customary to use a double 
subscript, the first one for the row and the second one for the column If a 
matrix is denoted A, then its elements may be denoted a,„ where t shows 
the row and j shows the column at the intersection of which the element a„ 
is found Thus, m Table la the element an = 3 and au — Q 

In developing the theory of matrices it is desirable to exhibit the ele¬ 
ments as shown in Table 2 The elements in the first row are an, aia, flu, 

, a ln , showing that the table represents a matrix of n columns The ele¬ 
ments of the first column are an, a 2 i> flan • - > a «a, showing that the table 
represents in rows The geneial element in this matrix A is a,,, where i 
takes the successive values 1, 2, 3, . , m, while j takes the successive 

values 1, 2, 3, ,n The first subscript refers to the row; the second sub¬ 
script refers to the column 

Table % 

<?n ^12 #13 din 

<Z 2 I ^22 O23 

&31 O32 033 

o»> 

Owl O !n 3 O mn 

The conventional repiesentation of a matrix is shown in Table 3, where 
the rectangular arrangement of numbers is inclosed by double vertical lines 
on the left and on the right sides of the rectangle It is also customary to 
denote specified matrices with letteis Thus, the matrix of Table 3 might be 
conveniently designated A or any other letter A matrix A might also be 
designated by its general element a„. 

Table 8 

2 15 

4 8 3 

2 0 7 
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If the successive rows of matrix A are written as successive columns of a 
new matrix, the new matrix is called the transpose of A It is denoted A\ 
Table 4 shows a matrix A and its transpose A' 


2 3 15 

Table 4 

2 1 0 

16 0 0 


3 (> 2 

0 2(17 


i o o 

A 


5 ‘1 7 

A' 


Determinants 

One particular interpretation of a square matux is called a ilr/i i mutant 
This interpretation of a squaie matnx probably had its ongin in the' pi net 1 - 
cal work of solving simultaneous equations, and it is indicated by single 
vertical lines on the left and on the right sides of a squaie table It is illus¬ 
trated in Table 5 Table S is called a matnx, while Table 5, which implies a 
particular interpretation, is called a dctnmmant A dcteiminanl is always 
square Hence its ordoi is n, in which n is the number of rows or the number 
of columns 

Tabic 6 
2 1 r> 

4 8 3 
2 0 7 

The diagonal from the upper loft corner to flic lowoi light eornei of a 
determinant is called the principal diagonal In Table o the pimeipnl diag¬ 
onal contains the elements 2, 8, 7 The other diagonal from the lower left 
corner to the upper light corner is called the secondary diagonal 

In many problems it is convenient to assign a plus sign and a minus sign 
to alternate cells m a dotcimmant A convenient rule is to designate the 
upper left cell as positive and all other cells as alternately negative and 
positive, as the cells can be moved over by the castle in a chess game This 
sign arrangement is illustrated m Table, (1 for a determinant, of oidei 


Table 0‘ 

+ — -J- — -1- 

- + - -h - 

+ “ + — -|- 

- -f- - + - 

~l" — + — -h 
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Notice that the cells m the principal diagonal are all positive and that lines 
parallel to this diagonal are alternately negative and positive The sign is 
positive for an even number of steps from the upper left cell, and it is nega¬ 
tive when the number of steps is odd The sign of a cell determined m this 
manner may be called the position sign of the cell If the general element of 
the determinant is denoted a,„ then the element with its position sign may 
be conveniently denoted (—l)*+’a„ When the exponent (Vi .?) is odd, the 
sign of the cell is negative; and when (i+j) is even, the sign of the cell is 
positive 

The product of any n elements of a square matrix, selected with only one 
element from each of the n rows and only one element from each of the n 
columns, is called a term of the determinant of the matrix Table 7 is a deter- 

Table 7 

Oil #12 #13 

021 #22 O23 

0 31 O32 0*J 


minant of order 3. From this determinant six terms may be written These 
are shown m Table S, in which the elements of each term are arranged in 
the order of their columns 



Table S 


1) 

#u 

#22 

#33 

3) 

a n 


#23 

3) 

#21 

#12 

#S3 

4) 

an 

#32 

#13 

5) 

#ai 

#12 

#23 

G) 

#ai 

#22 

#13 


Each of these six terms is the product of three elements so selected that 
each term contains only one element from each row and only one element 
from each column If a square matrix is of order n, the total number of 
terms in its determinant is n The term that contains all the elements of 
the principal diagonal is called the leading term of the determinant. 

The sign of each of the n terms of a determinant can be ascertained m 
the following manner Let the n elements of each term be arranged in as¬ 
cending order according to columns, as shown in Table 8 This can evident¬ 
ly be done without affecting the numerical value of the terms Consider 
the fourth term as an example, and list the rows as follows 


2 3 1 
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Any interchange of two adjacent elements constitutes an inversion It may 
be illustrated by interchanging 1 and 3 The resulting arrangement is 

2 13 

If, now, the adjacent elements 1 and 2 aie interchanged, the arrangement 
becomes 

12 3, 

in which the rows are in consecutive order. 

The sign of a term of a determinant is positive if it represents an even 
number of inversions from the consecutive older of rows and columns. The 
sign of the term is negative if the number of inversions in the term is odd. 

Applying this rule to the six terms of Table 8, we have the Rame terms 
with proper signs as shown m Table 9 



Table 9 


1 ) 

■{“flu 

on 

flaa 

2 ) 

— an 

a 32 

fl23 

3) 

— 0,21 

a la 

flaa 

4) 

-{-flai 

A 32 

flu 

r>) 

■{-flai 

fll2 

flaa 

0) 

-flu 

«22 

fll3 


A complete definition of a determinant can now be given 
Definition - If a square table is used as a symbol of the. sum of n terms, 
each term being the 'product of n elements with only one element from 
each row and only one element from each column, the sign of each term 
taken positive or negative according as the term contains an even or an 
odd number of inversions, then the square table is called a determinant. 
Hence the sum of the six terms of Table 9 is implied by the determinant of 
Table 7 The determinantal interpretation of a square matrix is denoted by 
single vertical lines on the left and on the right sides of the square table, us 
shown in Table 7 

If a square mahix is denoted by a letter such as A, then the determinant 
of the matnx is denoted |A|. If A represents a number, then |A| means 
the absolute value, ignoring the sign of the number A It should be noted 
that a matrix is merely a rectangular table of numbers, and hence a mati ix 
has no numerical value But a determinant is, by definition, a sum of terms, 
and hence it has a numerical value If a matrix is denoted a,,, then its de¬ 
terminant is denoted | a„ \. 
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Consider the second-order determinant 

1 5 

8 3 

and the 2 = 2 terms that it implies. These are 1X3 and 8X5, in which the 
factors of each term are arranged in consecutive order by columns The rows 
of the term 1X3 are 1 and 2 Since these are m consecutive order, the sign 
of this term is positive Its value is therefore -t-(l)(3) = +3 The rows of 
the term 8X5 are 2 and 1 One inversion changes the order 2 and 1 into the 
consecutive order 1 and 2 Hence the sign of this term is negative. The de¬ 
terminant therefore has the numerical value +3 —40= —37. Any second- 
order determinant can be evaluated as follows. 

a d 

= ab — cd 

c b 

An arrowed minor of the matrix A is a determinant of order x which is 
formed by the intersections of any x rows and any x columns of the ma¬ 
trix A. If one or more columns of a determinant are eliminated and if the 
same number of rows are eliminated, the remaining cells constitute a minor. 
From the determinant of Table 7 nine second-order minors may be drawn 
A few of them are illustrated here: 

fl u a vi 021 023 O l2 Oi3 

* » 

021 022 031 «33 022 0 2 3 

If any two columns and any two rows are eliminated from the determinant 

of 1 able 7, there remains a 1-rowed minor which is a single element In this 

sense each element can be regarded as a minor of the determinant 

If corresponding rows and columns are eliminated, the remaining minor 
is symmetrically placed with regard to the principal diagonal, and it is 
called a principal minor In the determinant of Table 7, three second-order 
principal minors may be drawn These are 

a 'l ®12 Oil Ol3 a 22 023 

1 1 • 

021 ® 22 asi 033 a 32 a 33 

There are three 1-rowed principal minors 111 this determinant, namely, the 
three elements in the principal diagonal 
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If the row i and the column j which intersect in an element are elimi¬ 
nated from a determinant, the remaining (n— l)-rowed determinant is 
called the first minor of a„. This definition is illustrated with the determi¬ 
nant of Table 5 The second row and the first column intersect in the ele¬ 
ment 4 If these two arrays are eliminated from the determinant, the re¬ 
maining 2-rowed determinant is 

1 5 

= 7 - 0 = + 7 . 

0 7 

This determinant, whose numerical value is +7, is the first minor of the 
element a 2 i = 4 in the determinant of Table 5 Let the first minor of the ele¬ 
ment a„ be denoted m„. 

In some problems it is convenient to refer to the minor rn„ witli the posi¬ 
tion sign of the element a„ This quantity is called the cofartor of «„ It is 
defined by the relation 

(cofactor of a„) = e„ = (-1 )"'m„ 

In Table 5 the cofactor of the element 4 is 

1 5 

(_l)i+i = - [7-0] = - 7 

0 7 

In the same table the cofactor of the element 3 is 

2 1 

(-1)2U = _ [0-2] = -t- 2 

2 0 

Hence the absolute values of the first minor of a„ and of its eofaelot aie 
identical I hey differ only in the manner of determining the sign If the 
position sign of the element a,, is positive, the first minor and the cofacior 
have the same sign If the position sign of a„ is negative, (hey have op¬ 
posite signs 

The numerical value of a determinant can be expressed convenient ly for 
some problems in terms of the cofactois Foi example, 

®U ®12 (Xl3 

c* 2 i CI 22 a 2 j = a urn + a 2 if> 2 i + a-nCai 

&31 (I 32 CI 33 
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The numerical value of a determinant is the weighted sum of the elements in 
any array , each element being weighted by its cofactor In the example, the 
determinant is expressed in terms of the elements of the first column. 

As a numerical example, the value of the determinant of Table 5 can be 
expressed as follows: 


2 15 
4 8 3 
2 0 7 

= 2(56-0) - 4(7-0) + 2(3-40) 
= 112 - 28 - 74 = + 10 


8 3 


1 5 


1 5 


-4 


+2 


0 7 


0 7 


8 3 


The numerical value of a determinant can be expressed as the summa¬ 
tion: 


(1) 




O'i]&i) , 




where the weighted sum may be taken over any column or any row. The 
following is an example of a fourth-order determinant, evaluated by the 
method of (1): 


|A| 



2 4 2 


4 10 


4 10 


4 10 

= +2 

6 14 

-3 

6 1 4 

+ 1 

2 4 2 

-1 

2 4 2 


0 2 3 


0 2 3 


0 2 3 


6 14 


2 4 2 
6 1 4 
0 2 3 


= +2 


1 4 

2 3 


-6 


4 2 
2 3 


+ 0 


4 2 
1 4 


= 2(3-8) - 6(12-4) + 0(16-2) 
= - 10 - 48 + 0 = - 58 


4 10 
6 14 
0 2 3 



1 4 


1 0 


1 0 

— +4 

2 3 

-6 

2 3 

+0 

1 4 


= 4(3-8) - 6(3-0) + 0(4-0) 
= - 20 - 18 + 0 = — 38 . 
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= 44 

4 2 

2 3 

-2 

1 0 

2 3 

40 

1 0 

4 2 

= 4(12-4) - 

- 2(3 

-0) 4 

- 0(2- 

-0) 

= 32 - 

- 640 = 

4- 26 . 



= 44 

4 2 

1 4 

-2 

1 0 

1 4 

46 

1 0 

4 2 


= 4(16-2) - 2(4-0) 4 6(2-0) 
= + 56 - 8 + 12 = 4 60 . 


Hence 

| A | = (+2)( — 58) - (3)( — 38) + (1)(426) - (l)(+60) - - 36 . 

For every element a„ in the square matrix A there is a corresponding 
minor m„ and a corresponding cofactoi e„- Let M be the square matrix 
with elements m„, let E be the square matrix with elements e<*; and let F be 
the transpose of E Then the square matrix F is called the adjoint of A. 
Its elements may be denoted/,, = e,. 

These definitions are illustrated in the following numerical example: 



2 

1 

5 


56 

-22 

-16 


4 

8 

3 

= A 

- 7 

4 

2 


2 

0 

7 


-37 

14 

12 

56 

22 

— 

16 


56 

-7 

-37 

7 

4 

- 

2 

— M 

-22 

4 

14 

-37 - 

-14 


12 


-16 

2 

12 


= F = adjoint of A . 
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A square matrix A is said to be symmetric when A' = A Then we also 
have aj, = a tj . It is symmetric about the principal diagonal 

14 5 

4 2 8 =ft symmetric matrix 

5 8 3 

If a„ = —a„, so that the signs above the principal diagonal are opposite 
to the sigDS below the diagonal, then the matrix is said to be skew symmetric 
A matrix A is skew symmetric if A'= — A 

0-3 4 

3 0—5 = a skew symmetric matrix 

-4 5 0 

If all the principal minors of a matrix are greater than or equal to zero, 
then the matrix is said to be 'positive-definite. If, in addition, it is symmetric, 
it is a Gramian matrix. 

2 3-3 

2 4 2 =a positive-definite matrix. 

3 5 6 

5 10 13 

10 20 26 =a Gramian matrix 

13 26 36 

In some problems it is important to know the highest order of the non- 
vamshing minors. The highest order of the non-vanishing minors is called 
the rank of a matrix. The rank of Table 5 is equal to its order, namely, 3, 
because the determinant itself does not vanish. The determinant of Table 10 
does vanish, so that its rank must be less than 3 It contains second-order 
minors that do not vanish, and the rank of the determinant is therefore 2. 

Table 10 
10 8 1 

8 8 2 =0 
12 1 
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If the determinant of a matrix is zero, the matnx is said to be singular. 
If the determinant does not vanish, the matrix is said to be non-singular. 

If | a it | =0, then a„ is singular. 

If |a,) | 7^0, then a„ is non-singular 

Most of the theorems m the elementary theory of real determinants am 
concerned with the methods of ascertaining the numerical value of a de¬ 
terminant and with the operations that do, or do not, affect its numencal 
value The following theorems are useful in dealing with determinants: 

1) The value of a determinant is equal to that of its transpose 


«1 

«2 



b, 



bl 

h 


a 2 

b 2 



a x 

b, 

Cl 


CLi 

a 2 

an 


b 2 

a 

= 

b, 

h 

b> 

a 3 

b 3 

Cl 


Cl 

Cl 

Cl 


2) If any pair of parallel arrays of a determinant are interchanged, the 
absolute value of the deteiminant remains unaltered but the sign is re¬ 
versed. 



a L 

bi 

Cl 



h 

Cl 

+ 

Ql 

h 

C2 

= — 

«i 

b, 

Cl 


a 3 

hi 

Cl 


a 3 

b 3 

Cl 


3) If two paiallel arrays of a determinant are pioportional, the determi¬ 
nant vanishes 


ai 

fall 

- 0 , 

ai 

ai 

a 3 


ka 2 


bi 

b 2 

b 3 




frb, 

kb 2 

kb 3 


= 0 . 
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If jfc=1, then two arrays are identical and the determinant vanishes 

4) If a determinant has an array of ciphers, it is equal to zero 

a b c 

0 0 0 = 0 . 

d e f 

5) If each element of an array is multiplied by any factor, the value of 
the determinant is multiplied by that factor. 


CLi 

<2j 

CLs 

at 

a a 

as 

bi 

h 2 

b 3 = k 

bi 

b s 

h 3 

kci 

kc 2 

kc 3 

Cl 

c 2 

Cs 


This theorem is sometimes useful in reducing a determinant to simpler 
forms 


6 

9 

8 

1 

1 

4 

1 

1 

4 

12 

18 

4 = 6X9X2 

2 

2 

2 =6X9X2X2 

1 

1 

1 

,24 

27 

2 

4 

3 

1 

4 

3 

1 


6) If each element in an array is reversed in sign, the value of the de¬ 
terminant is reversed in sign 


<3l 

h, 

Cl 


— Oi 

hi 

Ci 

—ai 

hi 

Cl 

a 2 

h 

Cl 

= ~ 

— 02 

bi 

+ 

II 

S 



— Cl 

a 8 


C8 


— a 3 


Cs 1 

— CL 3 

h 8 

Cs 


7) If an array contains no zero elements, all the elements of the array 
may be made unity by means of multiplying factors 


3 4 6 


1 1 1 


1 1 1 

2 8 8 

= 3X4X6 

1 2 1 

- 3X4 

4 12 8 

6 7 9 


2 i | 


2 I § 


111 111 
= 3X4X4 1 3 2 = 3X4 1 3 2 = —36 

2 i „ 9 


8 7 6 
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8) Every determinant can be expressed as a sum of two determinants 


(fll +p) 

hi 

Ci 


Oi 

hi 

Cl 


V 

hi 

Cl 

(U2 + ?) 

bi 

c 2 

= 

U 2 

h 2 

C 2 

+ 

Q 


c 2 

(o 3 +r) 

&3 

c 3 


o 3 

bz 

C 3 


T 

bz 

Cz 


9) The value of a determinant remains unaltered if each element in any 
array is augmented by a multiple of the corresponding element in a parallel 
array 


(oi+A:6i) hi Ci 


di bi Ci 


b i hi Ci 


b\ Cx 

(a 2 +H>2) C 2 

— 

#2 &2 

+ k 

hi bi Ci 

= 

02 ^2 C 2 

(a 3 +/eb 3 ) h 3 Cz 


(l* &3 ^3 


bz bi Cz 


a% b a Cj 


The second determinant m the right member vanishes because two columns 
are identical. 

10) If all the elements of a determinant on one side of the principal diag¬ 
onal are ciphers, the determinant reduces to the leading term. 


5 12 


5 12 


3 12 


2 12 


6 3 3 

11 3 8 


6 3 3 

5 0 5 


3 3 3 

0 0 5 


0 3 3 

0 0 5 

= 2X3X5= +30 


Matrix multiplication 

Consider the three simultaneous equations. 


2/i 

+ 2y 2 


= Xl 


3 Vi 

+ 2/s 

= x 2 

.2 yi 

+ 2i /i 

+ 

= X3 


The equations (2) are written in the expanded notation If the x’e are known, 
the equations may be solved numeneally for the y’s. As an example, let 
®i = l, X 2 = 3, X} = 2 Solving the equations simultaneously, 


Vi -A , 


2/2 = \l , 


2/s = A • 
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In some pioblems it is convenient, as well as clarifying, to represent a set 
of simultaneous equations m the rectangular notation which is illustrated in 
Table 11 Here the coefficients of the three simultaneous equations have 

Table 11 


k k 


1 

2 

0 


2/i 



0 

3 

1 

X , 

2/2 

= . 

Xi 

2 

2 

4 


2/3 




A y x 


been arranged in the form of a matrix that may be denoted A The y’ s have 
been arranged in a vertical column in a matrix denoted y The x’s are also 
arranged in a vertical column matrix denoted x The matrix A is of order 
3X3, while the matrices x and y are both of older 3X1 

A matrix that consists of a single column will be called a column vector 
A matrix that consists of a single row will be called a row vector The vec¬ 
torial terminology is probably due to the fact that the elements in any array 
may be regarded as the Caitesian co-ordinates of a point in a space of as 
many dimensions as there are elements in the array. This point, together 
with the origin, determines a direction in space In this manner any array 
of a matrix can be given a vectorial interpretation 

The three matrices A, y, x, of Table 11 may be regarded as symbolizing 
the simultaneous equations (2) This necessitates that a particular opera¬ 
tion be implied by the adjacent matrices A and y In order that these ma¬ 
trices shall symbolize the simultaneous equations, the following lule must 
be implied in Table 11 

The first equation m (2) can be produced from the matrices by writing 

3 

(3) Un2/u + ai2?/2i 4- ai3i/3i = Qi,2/,i = ain 

J = 1 

In performing this operation with the matrices, a row of the first one is 
associated with a column of the second one The first equation of (2) calls 
for the cross -products of the corresponding elements of the first row of A and 
the first column of y (In the present problem, y has only one column ) The 
cross product is illustrated by (3) 

The second equation (2) is produced by performing the same row-by- 
column multiplication, using the second row of A and the first column of y 
Then we have 

3 

a 2\Vu + 0222/21 + 0237/3! = 2 a-ziVn = *2i . 


(4) 
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The third equation (2) is produced by a similar operation on the third 
row of A and the first column of y. The sum of the three products is record¬ 
ed in the third row and first column of x The equation is 

3 

(5) #31?/ll + #321/21 4- #aal/31 — / ( #3)2/)! = ^31 

j-i 

The three equations may be written in the more condensed form 

a 

( 6 ) , a uVik ~ x ' k 

1 =1 

This interpretation of two adjacent matrices is called matrix multiplication 
In the present problem k=l, because y has only one column Since 1 can 
take three different values, namely, t = l, 2, 3, the equation (6) represents 
all three of the simultaneous equations in summational notation. Table 11 
represents the rectangular notation The three equations (2) may also bo 
represented conveniently in the still more condensed matrix notation, 
namely, 

(7) Ay — x, 

which is a matrix equation. The operation specified by this matrix equation 
is that if the matrix A is multiplied by the matrix y, row-by-column, the 
matnx product is another matrix, namely, x This is an exceedingly power¬ 
ful method of handling sets of equations, because many otherwise tedious 
numerical operations can be shunted, so that the calculations are performed 
only on a final set of matrices rather than on many intermediate stops Still 
more important is the fact that significant relations m a problem are con¬ 
spicuous m the matrix notation but they may be obscure when the prob¬ 
lem is handled in expanded algebraic or numerical form 

Table 12 

3 * l 

t II 2/i th Vi 11 10 2 k |l x, x a Xj l| 

?/' > 2 3 2 x' 

0 14 

A' 

The same set of simultaneous equations (2) may be represented by the 
matrix multiplication shown m Table 12 The multiplication of the first row 
of y' (y' has only one row) and the first column of A' reproduces the first 
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equation of (2), It should be noted that the matrices of Table 12 are the 
transposes of the matrices of Table 11. The transpose of a column vector is 
a row vector with the same elements, 

The summational notation for the matrix multiplication of Table 12 is 


(8) 



= x kt 


The general element of A m Table 11 is a„. Hence the general element of A' 
in Table 12 is a„ The general element of y is y,k, so that the general ele¬ 
ment of y' is y k ,. The matrix equation for Table 12 is 

(9) y'A' = x' , 


which represents the same set of equations as (7) 

In order to multiply one matrix by another, the number of columns of 
the first one must be the same as the number of rows of the second The col¬ 
umns of A in Table 11 are represented by the subscript j, and this is also the 
subscript for the rows of y If the subscripts for the first matrix are i and j 
and the subscripts for the second matrix are j and k, then the j subscript is 
ehminated from the matrix product which has the subscripts i and k The 
same rule can be verified in the matrix multiplication of Table 12, where the 
subscnpts of the first matrix are k and j and those of the second matrix 
are j and i Eliminating the middle subscript j, which is common, the ma¬ 
trix product has the subscripts k and i. 

The matrix equations (7) and (9) illustrate the following matrix theorem: 
Theorem: The transpose of any product of matrices is the product of their 
transposes m reverse order. 

Hence, if AB = C, it follows that B'A' — C' Applying this theorem to the 
present example, we have, by Table 11, Ay=x, and, by the theorem, y'A' 
, which is the matrix equation for Table 12 
If the %’s are known in (2), then the y’s may be found Let the y’s be 
expressed as linear functions of the z’s in (10). 


( 10 ) 


Z i +Z3 = V] , 

2Zi +83 = y 2 , 

= Vi . 


Zl +2»2 
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This set of simultaneous equations is represented in Table 18. If the three 

Table IS 


10 1 


Zi 


2/i 

0 2 1 

• 

Zj 

= 

Vi 

1 2 0 


Zs 


2/1 


B z y 


matrices of Table 18 are denoted B, z, and y, we can represent the three 
equations (10) in the single matrix equation, 

(11) Bz = y . 

Since the y’s are known, the values of the z’s can be determined Substitut¬ 
ing the known values of the y’s m (10), we find that 

2 2 = ih , Zs= - | 2 4 

Equation (7) shows that the x’s can be expressed linearly in terms of 
the y’s. Equation (11) shows that the y’s can be expressed linearly in terms 
of the z’s. It is desired now to express the x’s directly in terms of the z’h 
without the intermediate y’s This can be done From the equations 

(7) Ay = x , 

(U) Bz = y , 

it follows that 

A(Bz) = x , 

ABz = x 

Let AB = C. Then 

(12) Cz = x 

In order to express the x’s in terms of the z’s, the matrix product AB**C 

must be determined numerically. This matrix product is shown graphically 
in Table U. Consider the first row of A and the first column of B. The cross 
product is 


( 1 )( 1 ) + ( 2 )( 0 ) + ( 0 )( 1 ) = + 1 . 
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This is therefore the element in the first row and the first column of the 
matrix product C. Consider, as another example, the second row of A and 
the third column of B The cross product is 

(0) (1) + (3) (1) + (1)(0) = + 3 

This is the element e I3 m the matrix C 

Table 14 


1 

2 

0 

1 

0 

1 


1 

4 

3 

0 

3 

1 

0 

2 

1 

= 

1 

8 

3 

2 

2 

4 

1 

2 

0 


6 

12 

4 


A B C 


Since the numerical values of the x’s, the y’s, and the z’s are known, the 
matrix equation (12) may be tested graphically, as shown in Table IB As a 


Table 15 


1 

4 

3 

“Ti 


1 

1 

8 

3 • 

TT 

= 

3 

6 

12 

4 

_ » 

1 4 


2 


C e 


sample check, consider the second row of C and the first column of z It 
should reproduce the value a; 2 i = 3 


(D(-A) + (8)(,L) + (3)(-,V) = + 3 


Table 14 shows the matrix product AB = C. If the order of the matrices 
i and B is interchanged m tins multiplication, a different product is ob¬ 
tained This is readily verified numerically m Table 14, and it illustrates the 
principle that if AB-C, then, in general, BA^C Matrix multiplication is 
not commutative In matrix algebra it is essential to note the order of the 
matrix factors because the order is not arbitrary, as in ordinary algebra, 


The following is an example 
the transposed forms of these 


of matrix algebra If, instead of (7) and (11), 
equations were used, we should have 


y'A' = 


(9) 

(13) 


z'B' = y' 
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Substituting (13) in (9), we have 

( 14 ) z'B'A' = x'. 

But 

(15) AB = C 

Hence 

(16) B'A' = C' . 

Substituting (16) in (14), we obtain 

(17) z'C' = x' , 

which could also be written directly as the transposed form of (12) 

In order that theie shall be a unique solution for the simultaneous equa¬ 
tions (2), the matrix A of the coefficients must be non-singular, l e , | A | XO 
This may be tested by trying to solve a set of non-homogenoous simultane¬ 
ous equations with coefficients whoso determinant does vanish 

The multiplication of matrices is associative. This is illustrated as fol¬ 
lows - 


(. AB)C = A(BC) = ABC . 


The matrix pioduct (AB) may be determined and then poalmultiplied by 
C, or the matrix product ( BC ) may be determined and then pr c multi plied 
by A. The product is the same. This principle can be extended to any 
number of matrix factors. For example, 

C ABC)D = (AB) (CD) = A (BCD) = A BCD . 

Note that the ordei of the matrix factors is retained. 

The sum, or difference, of two mXn matrices is the mX n matrix each of 
whose elements is the sum, or difference, of the corresponding elements in 
the given matrices 


1 

2 

+ 

2 

3 


3 

5 

3 

4 


4 

5 


7 

9 
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The components may be written in any order. 

A B — B A t 

(A + B) + C = A + (B + C) = A+ B + C 
If k and m are scalars, then 

kA + kB = h(A + B) ; 

kA -f mA =* (As + m)A . 

The multiplication of matrices is distributive 

A(B + C) = AB + AC ; 

(B + C)A = BA + CA 

It can be shown that the rank of a matrix product cannot exceed the lowest 
rank of any of the factors. Thus, if the ranks of matrices A, B, and C are 
2, 4, and 3, respectively, then the rank of the matrix product ABC cannot 
exceed 2. 

It is sometimes useful to know that the determinant of the product of two 
square matrices is equal to the product of their determinants The following is 
an example: 


Let \A\ = 


2 6 
4 7 


= - 10 , 


and let IB] = 


2 1 
3 4 


= + 5 . 


Then 


\A\ • |B| = |AjB| = 


22 26 
29 32 


= - 50 = (-10)(+5) 


In the matrix product AB, the matrix B is said to be premultiplied by the 
matrix A, or the matrix A is said to be postmultiplied by the matrix B 
The operation of multiplying one matrix by another can be summarized 
or mnemonic purposes in the diagram of Figure 1. This diagram shows 
that rows of the first matrix are associated with columns of the second 
matrix and that the middle subscript is eliminated in the product. If the 

r0W f. A 18 “os* multiplied with the kth column of B, the cross product 
is recorded m the cell ik of C 1 

muTtiZatio 11 ,! 0 ^ 11 ? h maS1 l al ^ profouild ln the Particular rules of matrix 
multiplication that have become conventional The row-by-column rule is 
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entirely arbitrary. It would have been possible to set up a column-by-row 
rule provided that the matrices had been so arranged that the rule would 
have reproduced the original equations which the matrix notation represent- 



Figvre 1 

ed, It would also have been possible to have a notation which implied that 
one matrix was on top of another, but this would not have been so conven¬ 
ient for writing habits that go from left to right 

Diagonal matrices 

In the manipulation of systems of equations there occurs frequently a 
type of matrix in which all of the elements are zero except the diagonal ele¬ 
ments A matrix in which only the elements of the principal diagonal are 
non-vanishing is called a diagonal matrix. The following is a diagonal ma¬ 
trix of order 4: 

a 0 0 0 

0 6 0 0 

= D = a diagonal matrix. 

0 0 c 0 

0 0 0 d 

It sometimes happens that all of the elements of a diagonal matrix aic 
identical Such a matrix is called a scalar matrix. The following is an ex¬ 
ample: 

k 0 0 0 
0 k 0 0 

= K = a scalar matrix, 

0 0 A; 0 
0 0 0 A; 
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When a diagonal matrix has unity in each diagonal cell, it is called a 
unit matiix or the identity matrix. The following is an identity matrix of 
order 4: 

10 0 0 

0 10 0 

= I = the identity mati ix 

0 0 10 
0 0 0 1 

The properties of diagonal matrices are very useful m handling sets of 
linear equations 

1) Premultiplication DA with a diagonal matrix D multiplies each row 
of A by the corresponding element in D 

2) Postmultiplication AD with a diagonal matrix D multiplies each 
column of A by the corresponding element in D 

p 0 ah ap bp 

0 q c d cq dq 

DA DA 

a b p 0 ap bq 

c d 0 q cp dq 

A D AD 


3) Premultiphcation or postmultiphcation with a scalar matrix K mul¬ 
tiplies all elements of A by the constant element of K This is a special case 
of the first two theorems The reason why premultiplication with a scalar 
matrix has the same effect as postmultiphcation is that if every row is mul¬ 
tiplied by a constant p, the effect is the same as if every column is multiplied 
by the constant p In either case every element of A is multiplied by p 


a 

b 

p 

0 

c 

d 

0 

V 

A 


K 


P 

0 

a 

h 

0 

P 

c 

d 

K 


A 


V 

0 

a 

b 

0 

V 

c 

d 

K 


A 



ap bp 

cp dp 
AK 

ap bp 

cp dp 
ICA 


I o b 
= V 

| c d 
pA 
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Since the effect of a scalar matrix is independent of its position before or 
after the other matrices in a matrix product, its constant element p can be 
used in the product instead of the scalar matrix K as shown in the third ex¬ 
ample This illustrates the following theorem 

4) If K is a scalar matrix, then its constant element p may be substituted 
for the scalar matrix in a product 

AK = KA = pA = Ap . 


A multiplier which is independent of the non-commutative rule of matrix 
algebra is called a scalar 

5) To multiply a matrix A by a scalar p, in either older, pA or Ap, is to 
multiply each element of A by p 

The identity matrix is a special case of the scalar matrix, and hence it is 
also independent of the non-commutative rule of matrix multiplication 


1 0 


a b 


a b 

0 1 

I 


c d 

A 


c d 

IA=A 

a b 


1 

0 


a b 

c d 


0 

1 


c, d 


A I AI — A 


6) To multiply a matrix A by the identity matnx, in eilliei order, AI or 
I A, is to reproduce the matrix A unaltered. 

AI = IA = A . 


The identity matnx 7 in matnx algebia corresponds to unity in ordinary 
algebra Hence the identity matrix is suppressed, just as unity is suppressed 
m ordinary algebra. 

1X5 = 5, 

1 X x = x , 

IA = A . 


The inverse 

If a matrix A is non-singular, ic, |A|?^0, then tlicie exists another 
unique matrix such that its multiplication by A produces the identity ma¬ 
trix. This other matrix is called the inverse of A, and it is denoted A -1 
Hence, if A is non-singular, 


AA~ l = I . 
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The inverse of A -1 is A, so that 

AA -1 = I = A _1 A . 

Consider the ordinary algebraic equation, 

ax = y . 

If it is desired to state x explicitly, the equation is ordinarily written as 

1 

x = - y . 
a 

If the given matrix equation is 

AB — C 

and if it is desired to write it explicitly for B, this cannot be accomplished 
by ordinary division. A matrix: is not a number but a rectangular table of 
numbers. There is an operation in matrix algebra which corresponds to 
division in ordinary algebra If both members of the matrix equation AB 
= C are premultiplied by A -1 , we have 


But 

Hence 

or 


A- l AB = A-*C 


A~'A - 1 . 
IB = A~ l C , 


B = A~ l C 


This is the desired form. This example illustrates the operation in matrix 
algebra which corresponds to division in ordinary algebra It consists m 
moving a premultiplying or postmultiplying factor from one member of the 
equation to the other member m the same relative position. This operation 
is illustrated in the following examples' 

If 

ABC = M , 


BC = A~ l M, 

C = B-'A-'M , 
CM - 1 = B-L4- 1 , 
CM- 1 A = S-i , 

M~'A = C-ig-i 


then 
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Since this operation is analogous to ordinary division, the inverse of A is 
sometimes called the reciprocal of A. 

The inverse of any product of matrices is the product of their inverses in 
reverse order. 

Let ABC = M , 

BC = A~ l M , 

C = B~'A~ l M , 

I = C-'B-'A-'M , 

M- 1 = C-'B-'A- 1 . 

But 

(ABC)-'- = M- 1 . 

Hence 

(ABC)- 1 = C-'B-'A-' 

A method of writing the inverse of a given matnx is as follows Let the 
given matnx be A with elements a„. 

1) Write the matrix M with elements ra„ which are the first minors of 
the elements a,„ 

2) Reverse the signs of alternate elements of M so that it becomes the 
matrix E with elements = (— 1 

3) Write the transpose of E, namely, E' = F, with elements /„ =e n The 
matrix F is the adjoint of A; 

4) Divide each element of F by the value of the determinant |/11 This 
is the inverse A -1 with elements 


The writing of an inverse will be illustrated by the numerical example of 
equations (2) The given matrix equation is 

Ay — x 

It is desired to find the inverse of A so that the equation 

y = A~'x 


may be wntten m numerical form. 
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1 5 

0 


o a 

1 -A, 1A | = +14 . 


2 2 

4 


10 

-2 

— 6 


8 

4 

-2 =M , 


2 

1 

3 

10 

2 

-6 || 


- 8 

4 

II 

c* 

2 

-1 

3 II 

10 

-8 

2 

2 

4 

-1 -F, 


- 6 

2 

3 

u - 

-A 

A 

A 

A 

—A =A _1 , l-A -1 ! = +iV 

— 

A 

A 

A 


It is of interest to verify numerically the matrix equation y = A ’a\ It is 
written in rectangular notation in Table 16. 

Table 16 


1 0 
T4 

— ft 

2 

T? 

i 

II II 

ft 

A 

_ 1 

IT 

• 3 

II 

“ ft 

ft 

3 

U 

2 

II All 


A- 1 


X 

V 


In a later section of this Mathematical Introduction will be found a prac¬ 
tical computing method for the inverse The method described above is not 
suitable for matrices of large order 


The characteristic equation 

The c haJftetonst ic equation is of considerable theoretical interest in fac¬ 
torm several of the fundamental factor problems. 
~Fd§ 4hS^¥Sas'(S\i^'^wscribed m this mtroduction In a more complete 
presentatu^^r^his subject, the characteristic equation should be 
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introduced with some geometric and other mterpietation, so that the sig¬ 
nificance of this equation might be apparent The relation of the charac¬ 
teristic equation to the problems of factor theory will appear in later 
chapters. 

If a constant 0 is added explicitly to each diagonal element of a square 
ma tnk A, the resulting matrix is called the characteristic matrix of A It is 
illustrated as follows 1 


a ii 

ai2 

ai3 


1 

0 

0 


(«n+0) 

a 12 

<Il3 

0,21 

&22 

a 23 

+ 0 

0 

1 

0 

= 

1X21 

(022+0) 

«23 


0*32 

A 

U33 


0 

0 

0/ 

1 


«31 

U32 

(A+07) 

(«33+0) 


Chaiactcnstic mahix of A, 


The determinant of the characteristic matrix is the characteristic determi¬ 
nant of A 

The expansion of a characteristic determinant of order r is a polynomial 
of degiee r When this polynomial is set equal to zero, the equation is called 
the cliaractenstic equation of A An example is the following equation 


(1+/3) 0 1 

1 ( 2 + 0 ) 1 
1 0 ( 2 + 0 ) 


= 0 


When the determinant is expanded, the characteiistie equation becomes 


0 3 + 50 2 + 70 + 2 = 0 

The coefficients of the expansion of a characteristic determinant can bo 
written m terms of the puncipal minors without expanding the whole de¬ 
terminant Let the characteristic equation be as follows. 


(18) 0 r + m 1 0 r_I + mvP r ~ 2 + • • • + vi r = 0 . 

Then the coefficient m, is the sum of all the a-iowed principal minors in A. 
The coefficient mi is the sum of all the 1-rowcd principal minors of A. These 
are 1, 2, 2, and the sum is +5 The coefficient rn 2 is the sum of all the 
2-rowed principal minors of A These are 


1 0 
1 2 


1 1 


M f.B. tAbr'ary Document 


= 1 , 


2 1 


= 4 



28 


MULTIPLE-FACTOR ANALYSIS 


The coefficient m% = 24-1+4 = +7. The coefficient m s is the sum of all prin¬ 
cipal minors of order 3. This is the determinant (A j itself, and its value is 
+2 The coefficient of the highest power of (3 is unity. These coefficients 
can be verified by expanding the determinant. 

Another method for finding the coefficients of a characteristic equation 
has been described by Paul Horst in the Annals of Mathematical Statistics 
Vol VI, No 2 (June, 1935). 

The summational notation 

If a set of n numbers is to be summed, the operation may be indicated in 
the expanded form, 

(19) *i + %2 + Xs + • • ■ + x n = y . 

The same operation may be indicated m the more condensed summational 
form, namely, 

n 

(20) = V , 

t=l 


m which the subscript i takes the successive integral values from 1 to n 
inclusive. ’ 

In statistical work it is pedantic to indicate the limits, because the sum¬ 
mation is over the entire population except in rare cases, which can be 
specially indicated It is acceptable practice in statistical work to write 2x 
without subscripts when the usual form of summation over the population 
is implied In factor analysis this simple and convenient notation becomes 
ambiguous because summation may be over the factors, over the popula¬ 
tion, or over the variables It is therefore advisable to adopt the unam¬ 
biguous double-subscript notation that is conventional in mathematics 

As an example, the sum of the elements in the first row of Table 2 can be 
written in the form 


an -J- an -f- a 13 -f- 


+ ain = 

= i 


Here it is the second subscript j, representing the columns, which is found 
m the summation sign. This means that the a ’ s are to be summed for all 

bv b?” 1 “ Tta Wb8ct “ ,t “ Th. summation ta them 

oy contmed to one row. 

the^m 0 ^ 1011 ° an ^ generaIlzed t0 re P^sent any row It then takes 


o.i + -+ a in = . 

1=1 
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This notation means that the o’s are to be summed for a fixed value of t, 
since i does not occur in the summation. The a’s are to be summed in one 
row i for all the column values of j from 1 to n 

By analogy, the sum of all the elements in a column j of Table IS may be 

represented as follows' 

m 

dll + H" + • ’ ’ + a ™i ~ ^ ( a ') • 

»-l 


Here it is the column j that is fixed because it does not occur in the summa¬ 
tion sign. The a’s are to be summed in some specified column j for all values 
of i from 1 to m. 

If it is desired to designate the sum of all the elements m the matrix, each 
of the m-row sums must be summed This involves a summation over both 
i and j. We then have 


Sum of all elements in A — f a., 


,-i j-i 


Since it does not matter whether the rows are summed before the columns, 
or vice versa, we have 


n m m n 

22 a,j= 

j=> 1 t=l x-1 ;■=! 


A matrix multiplication can also be designated by the summational no¬ 
tation. Consider the matrix multiplication AB — C of Table 17. 

Table 17 



j(n) 



*(p> 



k{p) 


Qil 

a\i 

Ctia 

bu 

bu 

biz 


Cu 

Cl 2 

f’la 

& 21 

0,22 

022 

iW b 2 1 

&22 

&23 

—*(wO 

C21 

C22 

C 28 

(In 


0-83 


^32 

^33 


C 3 I 

^32 

C 33 

an 

a 42 

®43 




C41 

C 42 

C43 


A 



B 



A/i = 

r; 


Let A be a matrix of order mX« with general element a„. Let B be a 
matrix of order nXp with general element Then the product A.B = C 
must be a matrix of order raXp with the general element r., k The middle 
subscript j for the general element disappears in the product, and so does 
the middle dimension n in the product (mn) (np) = (nip). In the example, 
m = i, n = 3, p = 3 
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The element cn is obtained by the cross multiplication of the first row of 
A and the first column of B In summational notation, 


n 

(21) Ouhii + ttu?>2i “I" <^13^31 = "y* ^ Cli ft, i = Cn 

1 = 1 


The second row of A and the first column of B 


( 22 ) 


fljibn -f - 022621 ~h Oijhsi — chjb]! C21 

1=1 


The ith row of A and the first column of B * 

n 

(23) a.ibn + + citibai ~ "yY di,bji ~ Ca . 

j=i 

The ith row of A and the fcth column of B. 

n 

(24) aiibik + + o-iibsi. — — c ik 

j=i 


The summation of (24) gives a single element in C If it is desired to indi¬ 
cate the sum of all the elements in the ith row of C, the summation will be 
over k with a fixed value for 1 . We have then 

p n p 

( 25 ) v 

*=1 ,=1 A =1 

Finally, if the sum of all the elements in C is to be indicated, the summation 
must also be made for all rows Then 

m p n m p 

<*> 

t=L *-l 3-1 i=l A=1 

The summation of (24) represents the multiplication of two matrices, a.,, 
and b,k, whose product is the matrix c,* It can be visualized in Figure 1. 
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As an example of manipulation with the summational notation, consider 
the first term of equation ( 12 -i) It is 


JL 

N 


N 


2 ;^ 


2 


in which it is desired to substitute 

1 N 

iVji = ! 

N jzLj mx ’ 

1-1 

in order to simplify the first term 

Since the order of summation is arbitrary, the order of the summations 
may be interchanged Then the first term becomes 

1 r 

m=»l i—l 

Since the subscript % does not occur in a%„ this factor is a constant during 
the summation over 1. Hence, it may be placed in front of the summation 
over 1 without altering the value of the first teim, which then becomes 

'/'■ a J”‘ ~ 2 ’~) x m, ■ 
m=*l 1 = 1 

But the reciprocal of is a scalar, and so it can be placed anywhere 111 the 
summation Changing its relative position, the first term becomes 

r N 

y a « i yv 

m=l j -1 


and now the substitution can be made more clearly. Suppressing the part 
which is equal to unity, the expression simplifies into 

2 * , a jm 

m=l 


These steps are more explicit than will oidinardy be found nccessaiy, but 
behandTe?^ taXtbm manner In whlch the sum mational notation can 
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Linear dependence 

A matrix of order mXn may be regarded as m sets of numbers with n 
numbers m each set Each row of numbers is then a set Table 18 is a matrix 

Table 18 


2 

3 

5 

1 

4 

2 

4 

C 

10 

2 

8 

4 

1 

jj 

5 


2 

1 

6 

9 

15 

3 

12 

0 


of order 4X6 In this table every row can be expressed linearly in terms of 
the first row By this is meant that for any row i there exists a constant c, 
such that 

(27) o„ = c,ai, . 

For the fourth row the constant c, is 3, so that each element in the fourth 
row is three times the corresponding element in the first row. When any 
row can be so expressed in terms of the first row, the rows are proportional , 
and it can be shown that the columns are then also pioportional If two 
rows are not proportional, they are said to be linearly independent, for one 
of them cannot be expressed linearly in terms of the other When each row 
can he expressed linearly in terms of one row, the rank of the matrix is 1. 
This means that all second-order minors vanish. This fact is readily verified 
m Table 18 

The idea of proportionality can be generalized to two or more dimensions. 
An example of rank 2 is shown in Table 19 In this matrix any row can be 

Table 19 

7 5 4 6 7 2 

2 2 0 4 2 4 

5 4 2 0 5 4 

9 7 4 10 9 6 

expressed as a linear function of any two rows In this particular matrix 
there are no two dependent rows This requires that two constants c x and 
(not both zero) exist such that 

«.J = Cidi, 4- C 2 fl 2j , 

where the elements in the zth row are expressed linearly m terms of the 
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In the following example the two constants for the third row of Tabic 19 
are determined For the first two entries in the third row, 


(29) 


| 5 = 7ci + 2c 2 , 
{ 4 = 5ci + 2c 2 . 


Solving (29) simultaneously, wc find that e, = 1/2 and c 2 = 3/4 Testing this 
on the last column, as an example, 

(2) a) + (4)(t) = 4 . 


A different set of constants must be determined for each successive inde¬ 
pendent row. 

Since each of the rows in Table 19 can be expressed linearly in terms of 
the first two rows, it can be shown that the matrix of Tabic 19 is of rank 2 
This implies that all third-order minors vanish. As an example, the fol¬ 
lowing third-order minor of Table 19 vanishes. 


5 6 7 

4 6 5 

7 10 9 


= 0 


It can be shown that if the rank of a matrix is i, then there exists u set 
of r columns, or rows, in terms of which each column, or row, can bo linearly 
expressed ' 

Geometric interpretation of linear equations 

The most frequent form of equation for a straight line m a plane is prob¬ 
ably 1 


(30) 


— mx + p , 


in which x and y are the two variables while m and p are two independent 
parameters. This agrees with the well-known fact that any two points de- 
termme a straight line. The multiplying constant m is the slope, and the 
a ive constant p is the y-intercept. In the present context it will be more 
use ul to begin with the equation of a straight line m the more general form • 


(31) 


a&i + a 2 X'i + k = 0 


This equation has two variables, a* and *, and three parameters, a„ a„ k 
b.nce only two points are needed to determine the line, it follows that the 
three parameters are not independent 
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Equation (31) can evidently be multiplied by any arbitrary constant 
without affecting its geometrical representation in the plane. Then (31) 
becomes 

(32) ca\X i -f- ca 2 %2 -{■ ch = 0 . 

Let the multiplier c be so chosen that the sum of the squares of the coefficients 
of the variables is equal to unity. Then 

(33) (cai ) 2 + (“ 2 ) 2 = 1 > 


or 

(34) 


_ 1 
Va\ + a\ 


Let (Mi = X], ca 2 = X 2 , ck = d 


Then 


(35) 


Xi.'Cj -f- X 2 x 2 -}- d — 0 , 


where 

(36) X? + XI = 1 • 

When the equation is written with this adjustment, it is said to be in 
normal form This definition is applied not only to the equation of a line 
In a plane, and to the equation of a plane m a space of three dimensions, 
but also to the equation of a hyperplane of (n— 1 ) dimensions in a space of 
n dimensions The number of dimensions of the space defined by equation 
(35) is (n— 1) where n is the number of variables. Hence equation (35) de¬ 
fines a space of one dimension, a line, in a space of n = 2 dimensions, a plane. 

When a linear equation is m the normal form, the parameters have in¬ 
teresting meaning The parameters Xi and X 2 are the direction cosines of the 
normal to the linear space which is defined by equation (35); and the 
parameter d is the distance from the origin to the same linear space The 
normal to the line makes cos _, Xi with the Xi-axis and cos _ 1 X 2 with the .r 2 - 
axis The direction cosines of a space are the cosines of the angles that its normal 
makes vnth the Cartesian co-ordinate axes. In the present case the space is a 
one-dimensional space, namely, that which is defined by equation (35). In 
order to avoid ambiguity, the normal is taken positive on the side which 
contains the origin Equation (35) may be interpreted geometrically as 
defining a space of one dimension whose normal has the direction cosines 
Xi and X 2 and which is distant d from the origin. 
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If the parameter d vanishes, then the space which is defined by the equa¬ 
tion contains the origin. In equation (35) the line contains the origin if d 
is zero. 

Equation (35) locates a one-dimensional space (a line) in a two-dimen¬ 
sional space (a plane) If a new variable, Xs, is added, the equation takes 
the form 

(37) XjZi -I- -f- Xsrra + d = 0 

It defines a space of two dimensions (a plane) in a space of three dimensions 
with three orthogonal axes Here, as before, if the equation is in normal 
form, then d is the distance of the plane from the origin, and the three coeffi¬ 
cients Xi, X 2 , X 3 , are the direction cosines of the normal to the plane They are 
the cosines of the angles that the normal makes with intersecting lines that 
are parallel to the Xi, Xi, and xa axes, respectively 

The direction cosines have the property that the sum of their squares is 
unity. In equation (35) the line is defined if the parameter d and one of the 
direction cosines are given In equation (37) the plane is defined by its dis¬ 
tance from the origin and any two of its direction cosines The third direc¬ 
tion cosine can be found from the fact that the sum of the squares of the 
direction cosines equals unity If d vanishes in (37), the plane contains the 
origin A plane through the origin is therefore defined by its direction co¬ 
sines, which are the direction cosines of its normal 
An equation of the same form in n variables is 

(38) Xi^i -f- X 2 a ; 2 -(- X3IC3 -)- . -4- Xnir,, -f- d — 0 . 

It defines a hyperplane of (n — 1) dimensions in a space of n dimensions If 

(38) is in normal form, d is the distance of the hyperplane from the origin. 
The X’s are the direction cosines of the normal to the hyperplane and hence 

(39) 2 X * = 1 ' 

l=»l 

In the present factor analysis the hyperplanes of primary interest contain 
the origin, so that the parameter d vanishes Then 

(49) Xi#] + X 2 IE 2 + X 3 X 3 -f- . . . -f- \„x„ = 0 . 

The n values of X, are said to be the direction cosines of the hyperplane L 
which is defined by its normal A. 
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A matrix may be given a geometric interpretation Let the matrix A be 
of order mXn. Then the n elements of each row may be regarded as the 
Cartesian co-ordmates of a point in n dimensions. Since there are m rows, 
the matrix may be thought of as defining the positions of m points, one for 
each row, m a space of n dimensions. Table 20 is of order 6X3. It can there¬ 
fore be regarded as de finin g the positions of six points in a space of three 
dimensions 

Table SO 
1 2-4 

-4 1 -11 

-2 -a 5 
3 5 - 9 

| 4 -2 14 

0 1-3 

Let the rank of an mXn matrix A be r. Then it can be shown that the m 
points are contained in a space of r dimensions which also contains the 
origin. The rank of the matrix of Table 20 is 2. Hence the six points should 
he in a plane which contains the ongin. The equation of such a plane is 

(41) XjXj + -f- X 3 r 3 = 0 , 

in which the x’s are the three co-ordinates of each of the points m the plane 
and the X’s are the direction oosmes of the plane. The X’s are not independ¬ 
ent parameters because of the conditional equation (39). Hence any two X’s 
define the plane. These may be found by any two of the six points which are 
not collinear with the origin, Since no two rows of Table 20 are propor¬ 
tional, no two of the pomts are collinear with the origin. If two such points 
were found, then these two pomts would define a line through the origin and 
not a plane 

Substituting the x’s of the first two points of Table 20 m (41), 

(42) Xl + 2X2 ~ 4 * s = 0 ' 

. - 4Xi + X 2 - llXj = 0 

Solving for Xi and X 2 in terms of Xa, we have 


f Xi — — 2Xs , 


(43) 


X 2 = 3Xa . 
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Substituting (43) in (39), 

(44) 4X“ + 9X1 + XI = 1 , 

or 

Xs = ~ , 

V 14 

and hence 

X _ 2 

XI “ 1/14 ’ 

X, = » 

V 14 

The equation of the plane in normal form is therefore 
2,3,1 

(45 ) __ n;El + _ n;C2 + __ a;3 = 0 

All of the four remaining points must lie in this plane, since the rank of 
Table 20 is 2 The three coefficients are the direction cosines of the plane, 
1 e , the direction cosines of the normal to the plane 

The distance of a point from the origin is V 2 a: 2 , where the x's are its 
co-o rdinates For e xample, the distance of the fourth point from the origin 
is l/ 3 M- 5 2 +( — 9 ) a = 10 . 72 . if the sum of the squares of the co-ordinates of 
a point is equal to unity, then the point is at unit distance from the origin 
Each point may be interpreted as defining the terminus of a vector from 
the origin. The scalar product of any two vectors is hji 2 cos <t>, where hi and 
h 2 are the lengths of the vectors and <j> is their angular separation. If the two 
vectors are of unit length, then the scalar product is the cosine of the angular 
separation. It can be shown that the scalar product of two vectors can be 
expressed in the form 


n 



J-l 


where 1 and l refer to points (rows) and,; refers to co-ordinates (columns). 
For example, the scalar product of the vectors defined by the second and 
fourth points of Table 20 is 


(-4)(3) + (1)(5) + (—11)( —9) = + 92 
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If the sum of the squares of the co-ordinates of each point is equal to unity, 
so that the points lie at unit distance from the origin, then the scalar prod¬ 
uct, or cross product, is the cosine of the angular separation of the vectors 
at the origin 

Orthogonal transformations 

The three simultaneous equations (2) may be regarded as representing 
the three co-ordinates of a point x (x h x%, x 3 ), the three co-ordinates of a 
point y (y lt y 2 , yfj, and the law by which each point y is transformed into a 
corresponding point x. For every point y, there exists some other point x 
whose co-ordinates can be found by (2) when the co-ordinates of y are 
known. This relation is called a linear transformation by which the points y 
are moved to the corresponding points x. Every pair of the corresponding 
points is related by the linear transformation (2), The transformation is 
called linear when the equations by which the x’s can be found from the 
y's are of the first degree, as is the case in (2). 

If the transformation is of such nature that the x’s can be obtained 
from the y’s by merely rotating the co-ordinate axes, then the ti ansforma- 
tion is an orthogonal transformation or rotational tumsformation In order 
that a transformation shall be orthogonal, it is evidently necessary that the 
x’s be at the same distance from the origin as the corresponding y’s because 
the distance of a point from the origin remains invariant when the co-ordi¬ 
nate axes are rotated It is also necessary that the angular separations, or 
scalar products, be invariant, because the configuration of the points is not 
altered by rotating the co-ordinate axes. 

The matrix A of Table 11 is called the matrix of the ti ansfoi mahon when it 
is regarded as the relation by which the points y arc changed into the 
points x. A linear transformation is represented in the more general form 
by the square matrix of Table SI It can be shown that a square matrix is 

Tabic SI 

On a u a n ai„ 

an a, 2 a 21 o 2 „ 

flai Ok a 35 Oa„ 

. d,/ 

dm a„ j a„ j a„„ 

orthogonal, i e., that it has the effect of rotating a set of points y into a set 
of points x, with the same configuration as the y’s. if it satisfies the following 

rtnnrhr.i one ■ 
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1 ) The sum of the squares of the elements in each row is equal to 
unity, 1 e , 

n 

(46) ^ = 1, 

3^1 


and 

2) The cross product of every pair of rows, i and Z, is equal to zero, i e , 

n 

(47) ’^a l ,a h = 0 , 

j=i 

when 17*1 It is immaterial whether the rotation is conceived as a lotation 
of the orthogonal co-ordinate axes in a fixed configuration of points or as a 
rotation of the configuration in a fixed reference fiame of the co-ordinate 
axes The lesult is the same 

The two conditions, (46) and (47), are of such frequent occurrence that 
it is sometimes convenient to combine them in a single statement This can 
be done by writing the conditions in the more condensed form 


(48) 8ii , 

3 = 1 

wheie the symbol 5,; is known as Kioneckei’s delta It is defined as follows: 

d lL = +1 when i = l, 

5,i = 0 when i l 

It can be seen that with this definition of 5,(, the single statement (48) 
covers the two statements (46) and (47) 

If the matrix of a transformation is squaic and if it satisfies (48), then the 
following conditions are also satisfied 

3) The sum of the squaies of the elements in each column is unity, l.c , 

±4 = i 

4 = 1 


(49) 
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4) The cross product of any pair of columns is zero, 1 e , 

n. 

(50) ^a„a, k = 0 , J ^ k , 

where j and k refer to columns. 

5) The determinant of the transformation is ± lj i ® • 

(51) |A| = +1. 

If an orthogonal co-ordinate axis is reversed in direction, then the corre¬ 
sponding co-ordinate for each point is reversed in sign. If an odd number of 
orthogonal co-ordinate axes are reversed in direction in an orthogonal trans¬ 
formation, then the determinant of the transformation is equal to —1. If 
an even number of axes are reversed, the determinant is equal to -f-1. These 
two statements can be made with reference to the configuration. If the ro¬ 
tational transformation retains the configuration of the points, the determi¬ 
nant of the transformation is equal to +1 If the rotation involves a sym¬ 
metric distortion of the configuration, the determinant of the orthogonal 
transformation is equal to — 1 




Table US 



X\ X, 

y, r, 

Yi Yz 

a 

1 -2 

866 - 500 I a 

- 134 -2 232 

b 

2 3 

500 866 1 b 

3 232 1 598 

c 

-1 4 

c 

1 134 3 964 



G 


d 

5 -2 

d 

3 330 -4 232 

e 

4 1 

e 

3 964 -1 134 


A 


II 


Each column of an orthogonal transformation shows the direction cosines 
of one of the new co-ordinate axes, referred to the given co-ordinate axes. 

A rotation of the co-ordinate axes implies that the given configuration 
and the transformed configuration are contained in the same space. The 
rows of a transformation correspond to the dimensions of the given configu¬ 
ration, and the columns of a transformation correspond to the dimensions 
of the new configuration. If the transformation is merely a rotation of axes, 
it is evident that the matrix of an orthogonal transformation is necessarily 
square. If the matrix of a transformation is of order mXn where m^n, 
then the transformation cannot be orthogonal, since the number of dimen¬ 
sions of the given configuration and the number of dimensions of the trans¬ 
formed configuration are not the same, However, such a matrix may satisfy 
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condition (47), and it is then said to be orthogonal by rows If the condition 
is satisfied for columns, as m (50), instead of for rows, then the matrix is 
said to be orthogonal by columns. 

Table 22 is a numerical example of the rotation of five points in a plane 
A is a matrix of order 5X2 The orthogonal transformation G is of order 
2X2 The matrix product AG — B is of order 5X2, and it shows the co- 



Fiqure 2 


ordinates of the same five points with reference to the new rotated co¬ 
ordinate axes In F%gure 2 the five points have been plotted for the given 
orthogonal co-ordinate axes, X x and X 2 , that are implied in A. 

Let it be desired to rotate these axes through an angle <t> = 30° The usual 
formulae for rotation of axes* can be written in the form of a 2X2 transfor¬ 
mation as follows: 


cos 4 > 

— sin 

sin <t> 

COS (/> 


* W. F. Osgood and W C Graustein, Plane and Solid Analytic Qeometry (New York• 
Macmillan Co , 1929), p 220 
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The first condition (46) gives 

cos 2 <j> + sm 2 <t> — + 1 , 
and the second condition (47) gives 

cos $ sin ^ — cos </> sm <j> = 0 . 

Hence this is an orthogonal transformation in which the other properties 
may be readily verified 



substitution of 0-30 in the transformation produces the matrix G, and 

the multiplication AG produces B The numerical values in B may be 

lon= eC l m FigureS ’ where F 1 and F 2 have been drawn so that Y n OX x = 4, 

r ', F ® X “ m P ]le ’ the co-ordinates of the second point can be measured 

2 andYSu e°t 23 “? 1 6 ° f ° r the tW ° r ° tated axes - "luto they are 
2 and 3 for the two original axes This figure illustrates the geometric in- 
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terpretation of an oithogonal transformation The two columns of the trans¬ 
formation G show the direction cosines of the new orthogonal reference 
vectors Ti and Y 2 . 

Oblique transformations 

In Figure 2 the two co-ordinate axes Yi and I '2 aie orthogonal. If it is 
desired to define the n points with reference to oblique refeience axes, the 
transfoimation is effected in a similar way In Fig we 8 the same five points 
are plotted on the Xi and X 2 axes which aie implied in the given matiix A 
of Table 22. Here the new axes are oblique, Z\ is lotated 30° fiom A'], and 
Zi is rotated 45° from X 2 

In Table 23 are shown the numerical values for the conespondmg oblique 
transformation The matrix A contains the eo-oidinatcs of the given five 

Tabic S3 


Xi X2 Zi Zi Zi Zi 


a 

1 -2 

SWi - 707 a 

- i:u -2 121 

b 

2 3 

500 707 b 

3 232 707 

c 

-1 4 

c 

1 131 3 535 

d 

5 -2 

II <1 

3 330 -4 01!) 

e 

4 1 

c 

3 (Hit -2 121 


A 


All — C 


points in a space defined by the two orthogonal reference axes Xi and A '2 
The matrix II is a square matrix of order 2X2 It is the matrix of the oblique 
transformation Its columns show the direction cosines of the new oblique 
co-ordinate axes Z\ and Z 2 . These direction cosines may be verified 111 
Figure S 

It should be noted that the sum of the squares of each column of II is 
equal to unity Each column of II may be icgaided as defining a unit vec¬ 
tor m a space of two dimensions. The cioss pioduct of the columns of 
H is 

+ X2M2 , 

which is the cosine of the angle between Zn and Z 2 

The product AII = C shows the projection of each of the live points 011 
each of the oblique axes Z x and Z 2 This interpretation can be venfied by 
actual measurement on Figure 8. 
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A method of finding the roots of a polynomial 
Consider a polynomial of the type 


(52) f(J3) = CoP'+Cip- 1 + c# 2 + • ■ • Cr-iP + c r - 0 , 

where r is a positive integer, c a ^0, and Co, ft, . . , c r are real coefficients 

The r roots, / 3 , of this equation are desired Determine the upper and lower 
limits* of the roots of this equation Let a trial value of 8 within these 
limits be /S' If 0 ' is a root of the polynomial /( 0 ), then by the Remainder 
Theorem, f /(/S') = 0 The numerical value of /(/3') may be determined on an 
electric calculating machine by computing/(/3)/(/3—/3') by the process of 
synthetic division % Consider the sign of the numerical value of /(/3') and 
select a second trial root, designated /3", which will give /(/3") opposite in 
sign to that of/(/S') When two such trial values of /3 are found, there is at 
least one root§ between them Determine a third trial value, /S'", by linear 
interpolation between/(/S') and/Q3"). If the value /(/S'") =0, then /S'" is one 
of the r roots of the polynomial If /(/S'") ^ 0, interpolate for successive trial 
values until that value of /3 is found for which the remainder, /(/3), is zero 
to as many decimals as required 

Repeat this process for each of the r roots of the polynomial by taking 
trial values in other regions between the upper and lower limits of the roots. 
In the method of principal axes, a very useful first approximation foi each 

71 

root, /3 m , of the characteristic equation is — a® m for each column rn of F, 

J = 1 

when c 0 , Cj, , c r are all positive. 

Table 25 shows the application of the method of synthetic division in 
calculating one of the roots of the equation- 


(53) /3 1 + 6 965369,3 3 + 10 810494/3 2 + 5 407203/3 + 840052 = 0 


The upper limit of the roots of this equation is any positive number, the 


n 

lower limit is -7 965369. The first trial value /3' = —5 011317= — 

j-i 



as found in a prmcipal-axis problem Since/(/S') is negative, /3" = — 5 020000 
was chosen arbitrarily to secure a positive value of/(/3") Linear interpo¬ 
lation between /(/3') and /(/3") gave the third trial value, /3"' = - 5 019710. 


* L. E Dickson, First Course m the Theory of Equations (New York: John Wiley & 
Sons, 1922), pp 21-23 


t Ibid , p 12 


t Ibid , pp 13-15 


§ Ibid , p 67 
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Linear interpolation between and/(/3") gave the fouith trial value, 

—5 019712, for which the value of the polynomial is + 000018 
Hence, one root of this equation is —5.019712. 

The values of f(0) in each row of Table 25 were determined by the equa¬ 
tions m their corresponding rows of Table 24 In actual application, it is 
possible to carry out the r calculations in each row on a calculating machine 
without recording any of the values except the ?th one in the column 
headed /(/3). 

Table 24 


0< 


P 

P 

/03) 

CO 

Ci 

Cl 

Ci 

Ci 


Cld-Oo/3’ ~Cr 
Ci+Co/3" =ci' 
Ci+Co/3'" = ci" 

Ci+Co/3"" = cl’" 

Cj+ci/3' — cl 
c 2 +ci'(3"=ci' 
c 2 +c;"/S"' = cf" 
c 2 +cl'"p""=cl>" 

cs+<+)'=cj 
c 3 +c+3"=cj' 
c s +cT0'" = cf" 
c a +c'i'"i 3""=c,"" 

Ci+cis' =ci 
Ct+Cs/3" —c't 
Ci +c' s "0"' = ci" 
c t +ci'"tl""=cT' 


Table 26 


Tnal 

P‘ 

P‘ 

P 

P 

HP) 

'I mil f) 

1 0 

0 90630 !) 

10 810494 

5 407203 

840052 

1 


1 954052 

1 018120 

305081 

- 088800 

— 5 011317 

2 


1 945369 

1 044742 

162598 

023810 

— 5 020000 

3 


1 945659 

1 043850 

107379 

- 000142 

-5 0111710 

4 


1 945657 

1 043850 

167347 

000018 

— 5 019712 


A method of computing the square root on a calculating machine 
Newton’s* iterative method of determining square roots may be used 
very advantageously on the calculating machine, using Barlow’s Tables to 
determine the first tnal value of the square root 
Let N be the number whose square root is desired, and let t 0 be the first 
trial value for the VN derived from Barlow’s Tables Determine N/x a on 
the calculating machine, and record it 
Compute a new trial value, xi = .5(xo+N/a;o) by determining the cumula¬ 
tive sum of the two products, ( 5x 0 ) and [ 5(+/+,)] Determine N/x x If 
the two values x x and N/x x are the same to as many decimals as desired, 
then VN = xi = N/x\ If these two values differ, this process may be re¬ 
peated for as many trials as are necessary to find that trial value x which 
agrees with N/x to the required number of decimals. 

E T Whittaker and G Robinson, The Calculus of Observations (London: Blnckio & 
Son, 1926), pp 79-80 
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= 42 072 


2 d 

= 1/42,07 

m Barlow’s Tables = 

6 4861391 . 

N 

Xq 


= 

6 48644738 

Xi 

— 5xq 4 ** 


6.48629324 

N 

Xl 


= 

6 48629324 . 


The computation of an inverse 

The outline that was described in a previous section for computing an 
inverse is adequate for small matrices of older 2X2 or 3X3, but it is not 
economical for laiger matrices The outline of computations which will be 
described m this section is applicable to mail ices of any order, and the 
computational labor is not piohibitive even when the older is as large as 
10 or 12 * b 

The method is designed to hnd the inverse of a symmetric matrix which 
may be denoted M. If it is desired to find the invoiae of a non-syminetnc 
matrix P, one can proceed as follows • Compute the malnx PI” = ]\f, which 
is symmetric Find the inverse M~ l of the symmcliic matn\ M by the 
method to be described here Then compute the matrix product 
We then have 


(54) ( M~ l P )' = [(PP')-ip]' 

= [(P , )- 1 CP- 1 P)J' 

= [(POT 

= p~ i 

Hence the method can be adapted to find the inverse of a matrix which rs 
either symmetric or non-symmetric* 

0 i£ 3X3 ft r ! laVe , a T° ncal examplc - a itmlMT M of 

3X3 ' U 18 p!aoed in the u PP et left cornei of the tabic m the first 

by Ledyard Tucker Seealar^papir^'Metlrod^fo^jr^rf 1 m tIlm set ' tlon was designed 
Psychameinka, HI, No 3 (September,’ mti), ISSMjV "' ld,ngU '° ^veracofa Matrix,” 
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three rows of Section A The steps m computing the inverse aio then as 
follows, including check columns 1 


Table 20 


Section A Section B Section C 



1 

2 

3 




I 

II 

III 


I 

ir 

III 

1 

80 

48 


30 



80 

0 

0 


1 00 

0 

0 

2 

48 

80 


30 



48 

51 

01 


00 

1 00 

02 

3 

36 

36 


80 



30 

14 

06 


45 

27 

1 00 






Ch 

1 

04 

00 

67 

Ch 

2 05 

1 27 

1 02 

2 i 

1 64 

1 64 

1 

58 

2 t 

1 

04 

05 

07 

2 , 

2 05 

1 27 

1 02 


1 00 

0 


0 


1 

00 

- 00 

- 20 


1 25 

-1 18 

- 44 


0 

1 00 


0 



0 

1 00 

- 27 


0 

1 90 

- 41 


0 

0 

X 

00 



0 

0 

1 00 


0 

0 

1 52 






Ch 

1 

00 

40 

41 

Ch 

1 25 

78 

07 

s 2 

1 00 

1 00 

1 

00 

s 2 

1 

00 

40 

44 

2 j 

1 25 

78 

07 






Ch 

2 

64 

1 06 

1 . 11 

Ch 

3 30 

2 Oti 

1 00 

2 

2 64 

2 64 

2 

Cn 

1 00 

2 

2 64 

1 05 

1 11 

2 

3 30 

2 05 

1 09 







1 

25 

1 90 

1 52 







(Jlqm) 










iU“i 



1 00 

- 60 

— 

29 


1 

25 

0 

0 


2 09 

-1 00 

- 41 


0 

1 00 

— 

27 


-1 

18 

1 90 

0 


-1 06 

2 07 

- 41 


0 

0 

1 

00 


— 

44 

- 41 

1 52 


- 44 

- 41 

1 52 


1 Recoid the column sums, 2 1; as shown 

2 In the next three rows of A wnte an identity matiiv 

3. Record the column sums, 2 2 , for these three lows 

4 Record the total column sums, 2 = 2 i+2 2 

5 In column I of Section B copy the first column of Section A Ileie a 
differentiation is made between the anticipated column sum, ('ll, and the 
actual sum, 2 

6 Find the leciprocal, l/h u = l/ 80= 1 2, r > The entry, b u , is the first di¬ 
agonal entry in Section B It is the entry in the first column and in the 
hist row of B Reeoid this recipiocal in the bottom low of B as shown. 

7 Compute the values m the first column of Scttion (' Those entries aio 


(55) 


_ , / 1 \ 
r i i — e,i( I 
\On/ 
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8 Compute the second column of B by the formula 
bj2 = a 3 i — (bnCsi) i 
where cn= 60 is constant for the entire column 

9. Find the reciprocal 1/622 = 1 /51-L90 

10 Compute the second column of C by the formula 

(57) = b ’ 2 (b^) 

11. Compute the third column of B by the formula 

(58) 6j 3 = ®j3 — bjiCn — 35 , 

where c al = .45 and c 3J = 27 are constants for the entire column 

12. Compute the third column of C by the foimula 

(59) = h »{£) ’ 

where the reciprocal l/b»t vs a constant multiplier for the whole column 

Continue the process in the same manner so that there will be as 
many columns in B and in C as there are columns in A The generalized 
formulae are, then, 

- "-‘(c) 


(60) 

and 

(61) 


OjA. 


b,k = — [ijl Ckl + 6,2Ci2 + + l)CA.(A_ Ij] 


13. When the columns of B and of C have been computed and checked 
by the column sums, the desired inverse M~' is 


(62) 


Af-> = (b qm ){c qm Y , 


where (b gm ) is the squaie matux defined by the columns of B and the lows 
of the identity matrix m A , The square matrix (c gm ) is defined by the 
columns of C and the same rows which are defined by the identity matiix 
m A Note that it is the transpose of the matrix, which is used for the 
final equation for M~ l The computed inverse is shown in the tabic 

Linear dependence of vectors 

Consider two vectors A and B in Figure 4 . These two vectors define a 
plane, namely, the plane of the figure These two vectors are said to span 
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the plane of the figuie. By this is meant that any vector in that plane can 
be written as a linear combination of the given vectors A and B Any other 
vector in that plane, such as C, can then be written as the linear com¬ 
bination 

C = aA + bB , 

wlieic the small letteis, a and b, repiesent scalars and the capital letteis 
repiesent vectois Any vector C in the plane of the figure is detcimmod 
entirely by the numerical values of a and b Further, all linear combina¬ 
tions of A and B arc confined to the plane of these two vectors No vectoi 
outside of that plane can be described as a linear combination of A and B 



This reasoning extends to any number of dimensions Tlnee vectois span 
a three-dimensional space if they are linearly independent. The three vec¬ 
tois A, B, and C, in Figure 4 aie not linearly independent because any one 
of them can be descubed as a linear combination of the othei two If thiee 
vectors aie not coplanai, they are linearly independent Then they span 
a three-dimensional space Any vector m those three dimensions can be 
described as a lmcal combination of the three independent vectors 

The interpretation of linear dependence of vectois will be illustrated by a 
numerical example Let it be required that the vector C shall satisfy three 
conditions, namely, (1) that it shall he m the plane of the given vectors A 
and B, (2) that it shall bo orthogonal to B, and (3) that it shall be of unit 
length. 

The first condition is satisfied if we define C as a linear combination of 
A and B Then 


(63) 


V = aA + bB , 
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where a and ft aie two parameters which are to be determined by the other 
two conditions The vector <7 will be oithogonal to B if the scalar product 


(64) CB - 0 , 
and C will be of unit length if 

(65) C 2 = 1 . 
Substituting (63) in (64), we have the equation 

(66) M + bB)B = 0 


or 

(67) a(AB) + ftB 2 = 0 

Substituting (63) m (65), we have 


(68) aU s + 2 ab(AB) + b 2 B 2 = 1 . 

Equations (67) and (68) can be solved explicitly foi a and ft, but, in com¬ 
puting, it is usually simplei to wnte, fust, a vectoi C' which is oithogonal 
to B by satisfying (67) without restriction as to its length Let a — 1. 
Then ft = — (AB)/B 2 When such a vectoi C' has been determined, it 
can be normalized, and it will then satisfy both conditions 

As a numencal example, considei Table 87 of dnecfion numbers foi 
A and B m three dimensions Then A 2 = 0 69; B 2 = 0 83, and the scalar pi ocl- 


Table 27 



I 

II 

III 

A 

7 

4 

2 

B 

5 

3 

7 

C 

3325 

1795 

- 3146 

C 

6763 

3651 

- 8400 


A 3 = 69 

73* = 83 

{AH) = 61 

B(c') * = 2417 

V Recip = 2 0341 


uct [AB) =0 61 If we set a=l 00, then 6= — (AB)/B 2 = — 0 7349, and then 
the dnection numbers of C' can he wutten by (63), as shown in the thud 
low of the table The cioss product of lows B and C' vanish, and hence 
these two vectois are orthogonal The low C' is next noimalized, as shown 
in row C, and then both conditions are satisfied, namely, CB = 0 and C 2 = 1. 
If the direction numbers of C weie reveised in sign, they would define the 
vectoi C u as shown in Figure 4 This vector also satisfies the two conditions 
imposed They differ in that the scalar pioduct CA is positive, whereas 
CiA is negative 



CHAPTER I 

THE FACTOR PROBLEM 


On the nature of science 

This volume is concerned with methods of discovenng and identifying 
significant categories in psychology and in other sciences It is therefoie of 
interest to consider some phases of science m general that bear on the prob¬ 
lem of finding a methodology for a psychological science 

It is the faith of all science that an unlimited number of phenomena can 
be comprehended in terms of a limited number of concepts or ideal con- 
stiucts Without this faith no science could evei have any motivation To 
deny this faith is to affiim the pnmary chaos of natuie and the consequent 
futility of scientific effoit The constiucts in teims of which natural phe¬ 
nomena are comprehended aie man-made inventions To discover a scien¬ 
tific law is merely to discover that a man-made scheme serves to unify, and 
thereby to simplify, comprehension of a certain class of natuial phenomena 
A scientific law is not to be thought of as having an independent existence 
which some scientist is foitunate to stumble upon A scientific law is not 
a pait of nature It is only a way of compichcnding natuie 

A simple example is the concept "foicc ” No one has evei seen a force 
Only the movement of objects is seen The faith of science is that some 
schematic representation is possible by which complexities of movement 
can be conceptually unified into an order The enor of a lileial intoi pi ela¬ 
tion of a force vectoi as the pictorial representation of a conesponding 
physical entity is seen in the resolution of foi ces If a pai tide moves with 
umfoim acceleration in a certain direction, it is, of course, possible to de¬ 
scribe the movement by one foicc, or by two, or by tlnee 01 moie coplanar 
forces This lesolution of a movement into several simultaneous and .supei- 
lmposed movements is frequently done in 01 dei that a convenient, and habit¬ 
ual refeience frame may be letamed While the ideal constiucts of science 
do not imply physical reality, they do not deny the possibility of some de¬ 
gree of correspondence with physical leality But this is a philosophical 
problem that is quite outside the domain of science 

Consider, as anothci example, Coulomb's liivorse-squaie law of decimal 
atti action A postulated force is expressed as a function of the lmeai sepa¬ 
ration of the chaiges Now, if the charges weie to be peisomfied, they would 
piobably be much sui puscd that their actions weie being described in tei ms 
of their linear separations No one assumes that theie is a sti mg between the 


charges, but Coulomb’s law implies that the length of such a stung is to be 
used m our simplified scheme of comprehending^!^ 1 a,teji 
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is more likely that the whole space surrounding the charges is involved in 
the phenomena of attraction and that Coulomb’s law is a fortunate short 
cut for representing approximately a part of the phenomena that are called 
“charges” and “attractions ” It is not unlikely that all these entities will 
eventually vanish as such and become only aspects of an order more in¬ 
volved than Coulomb’s law implies but not so chaotic as to individualize 
completely every moment of nature 

A science of psychology will deal with the activities of people as its cen¬ 
tral theme A large class of human activity is that which differentiates in¬ 
dividuals as regards their overt accomplishments Just as it is convenient to 
postulate physical forces in describing the movements of physical objects, 
so it is also natural to postulate abilities and their absence as primary 
causes of the successful completion of a task by some individuals and of the 
failure of other individuals m the same task 

The criterion by which a new ideal construct in science is accepted or 
rejected is the degree to which it facilitates the comprehension of a class of 
phenomena which can be thought of as examples of a single construct rather 
than as individualized events It is in this sense that the chief object of 
science is to minimize mental effoit But m order that this reduction may 
be accepted as science, it must be demonstrated, eithei explicitly or by im¬ 
plication, that the number of degrees of fieedom of the construct is smaller 
than the number of degrees of freedom of the phenomena that the reduc¬ 
tion is expected to subsume Consider, as an example, any situation in 
which a rational equation is proposed as the law governing the relation be¬ 
tween two variables If three observations have been made and if the 
proposed equation has three independent parameters, then the number of 
degrees of freedom of the phenomena is the same as the number of degrees 
of freedom of the equation, and hence the formulation remains undemon¬ 
strated If, on the other hand, one hundred experimentally independent 
observations are subsumed by a rational equation with three parameters, 
then the demonstration can be of scientific interest The convincingness of 
a hypothesis can be gauged inversely by the ratio of its number of degrees 
of freedom to that of the phenomena which it has demonstrably coveied 
It is in the nature of science that no scientific law can ever be pioved to be 
right It can only be shown to be plausible The laws of science are not 
immutable They are only human efforts toward parsimony in the com¬ 
prehension of nature 

If abilities are to be postulated as primary causes of individual differences 
m overt accomplishment, then the widely different achievements of indi¬ 
viduals must be demonstrable functions of a limited number of reference 
abilities. This implies that individuals will he described in terms of a limited 
number of faculties This is contrary to the erroneous contention that, since 
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every peison is different from every other peison in the world, people must 
not be classified and labeled 

Each generalization m the scientific description of nature results m a loss 
in the extent to which the ideal constructs of science match the individual 
events of experience. This is lllustiated by simple experiments with a pen¬ 
dulum, m which the mass, the period, and the locus of the center of gi avity 
with leference to a fulcrum aie involved in the ideal construct that leads to 
experimental venfication But the construct matches only incompletely the 
coiresponding expenmental situation The construct says nothing about 
the rusty setscrew and other extraneous detail Fiom the viewpoint of im¬ 
mediate experience, scientific description is necessarily incomplete The sci¬ 
entist always finds his constructs immeised in the nrelevaneies of expen- 
ence It seems appropriate to acknowledge this characteustic of science, in 
view of the fact that it is a lather common notion that the scientific descrip¬ 
tion of a person is noL valid unless the so-called “total situation’’ has been 
engulfed A study of people does not become scientific because it attempts 
to be complete, noi is it invalid because it is lestncted. The scientific de- 
scnption of a person will be as incomplete from the viewpoint of common 
sense as the dcscuption of othei objects in scientific context 

The development of scientific analysis in a new class of phenomena usual¬ 
ly meets with resistance The faith of science that nature can be compre¬ 
hended in terms of an ordei acknowledges no limitation whatcvei as ic- 
gards classes of phenomena. But scientists aie not fiee from prejudice 
against the extension of their faith to i calms not habitually compiehendcd 
in the scientific order Examples of this resistance are numerous It is not 
infrequent for a competent physical scientist to declare his belief that the 
phenomena of living objects are, at least in some subtle way, beyond the 
reach of ligoious scientific order 

One of the forms in which this lesistance appears is the assertion that, 
since a scientific construct does not, cover all enumerable details of a class of 
phenomena, it is therefore to be judged inapplicable Since the analysis of 
cell growth by mathematical and physical principles does not cover evei y- 
thing that is known about cells, the biologist judges the analysis to be in¬ 
applicable Since no mathematical analysis that can be conceived would 
cover all the subtle mysteries of personality, this lealm is frequently judged 
to be outside the domain of rigorous science. But physical scientists accept 
rigorous scientific analyses about physical events that leave fully as much 
beyond the scientific constructs Every explosion in the world has been 
different from every other explosion, and no physicist can write equations 
to cover all the detail of any explosive event It is certain that no two thun¬ 
derstorms have been exactly alike, and yet the constructs of physics aie 
applied in comprehending thunder and lightning, without any demand that 
the detail of the landscape be covered by the same scientific constiucts 
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The attitudes of people on a controversial social issue have been appraised 
by allocating each person to a point on a linear continuum as regards his 
favorable or his unfavorable affect toward the psychological object Some 
social scientists have objected because two individuals may have the same 
attitude score toward, say, pacifism, and yet be totally different m their 
backgrounds and in the causes of their similar social views If such critics 
were consistent, they would object also to the statement that two men have 
identical incomes, for one of them earns while the other one steals They 
should also object to the statement that two men are of the same height 
The comparison should be held invalid because one of the men is fat and 
the other is thin This is again the resistance against invading with the 
generalizing and simplifying constructs of science a realm which is habitual¬ 
ly comprehended only in terms of innumerable and individualized detail 
Every scientific constiuct limits itself to specified vanables without any 
pretense of covering those aspects of a class of phenomena about which it 
has said nothing As regaids this characteristic of science, there is no differ¬ 
ence between the scientific study of physical events and the scientific study 
of biological and psychological events. What is not geneially understood, 
even by many scientists, is that no scientific law is evei intended to repre¬ 
sent any event pictonally The law is only an abstraction fiom the experi¬ 
mental situation No expenment is ever completely lopeated 

There is an unlimited number of ways in which nature can be eompie- 
hended m terms of fundamental scientific concepts One of the simplest 
ways m which a class of phenomena can be comprehended m teims of a 
limited number of concepts is probably that in which a linear attribute of an 
event is expressed as a lineai function of piimary causes Even when the 
relations are preferably non-linear and mathematically involved, it is fre¬ 
quently possible to use the simple linear forms as fust approximations A 
well-known example of this type of relation is that in which the chroma of 
a spectral color is expressed as a linear function of two aibitrarily chosen 
primaries If two spectral colors are chosen arbitrarily for use as primaries, 
it is possible to expiess any mteimediate color as a linear function of the 
two arbitrarily chosen primaries The coefficients of the two terms of this 
linear function represent the angulai sizes of the two sectors into which a 
coloi rotatoi is divided When the rotatoi is spun, the intermediate color 
is seen But here, as elsewhere in science, although the chroma of the re¬ 
sulting color is expressed in teims of the linear function of the arbitiary 
primal les, it docs not follow that the saturation and the gray-values are 
expressed by the same law There is still debate about which colors aie to 
be considered primary This question can be settled only by discovering 
that a certain set of primaries gives the most parsimonious compi ehension 
of some phase of coloi vision A parallel in the delineation of human traits 
is their description, in first appioximation, as linear functions of a limited 
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number of reference traits The final choice of a set of pnmaiy lcference 
traits or faculties must be made in teims of the discoveiy that a paiticular 
set of lefeiencc traits renders most parsimonious our comprehension of a 
great variety of human ti aits 

The purpose of factor analysis 

A factor problem staits with the hope or conviction that a certain do¬ 
main is not so chaotic as it looks The range of phenomena that is lepre- 
sented m any factor analysis will be refei red to as its domain. If a pai tieular 
investigation is limited to measurements in visual perception, it is likely 
that auditory effects will be outside of its domain The factorial methods 
weie developed pumarily for the puipose of identifying the pnneipal di¬ 
mensions or categones of mentality, but the methods aie gcncial, so that 
they have been found useful foi othei psychological pioblcms and m other 
sciences as well Factor analysis can be regarded as a gencial wientifio 
method Since the methods were developed especially for the solution of 
psychological pioblems and since the new methods have been used so far 
mainly on psychological problems, these will be used foi most of the ex¬ 
amples in this text Some of the pnneiples can be lllustiated to best advan¬ 
tage in teams of simple mechanical 01 geometrical examples; and these will 
be used occasionally, especially when it is de.suod to lllusti ale a logical pi m- 
ciple without involving the distractions of conlioverbad or nebulous sub¬ 
ject matter 

The factorial methods were developed for the study of individual differ¬ 
ences among people, but the individual diffeiences may be legalded as an 
avenue of approach to the study of the processes winch undoilie these dif¬ 
ferences If a process is mvai lant, in all its chai actciistica in an expoi linental 
population of individuals, then there exist no individual dilforences as ie- 
gards such a process, and it cannot be investigated by factonal means 

Thus, if we select an experimental population of individuals who aie all 
equally good or equally bad in some form of visual perception, then we can¬ 
not expect to identify or diffeientiatc such processes by factonal methods 
It is only to the extent that the individuals of an expenmcntnl population 
exhibit individual differences in a piocess and its effects that these effects 
can become accessible to investigation by factoiial methods 

When a particulai domain is to bo investigated by means of individual 
differences, one can proceed in one of two ways One can invent a hypoth¬ 
esis regaiding the piocesses that undoilie the individual diffoionce.s, and one 
can then set up a factonal expenment, or a more dueet Inbmatory expel 1 - 
ment, to test the hypothesis If no piomising hypothesis is available, one 
can repiesent the domain as adequately as possible in teims of a set of 
measurements or numerical indices and proceed with a factorial experiment 
The analysis might reveal an undeilying order which would be of great as- 
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sistance m formulating the scientific concepts covering the particular do¬ 
main In the first case we start with a hypothesis that determines the na¬ 
ture of the measurements that enter into the factorial analysis. In the sec¬ 
ond case we start with no hypothesis, but we proceed, instead, with a set of 
measurements or indices that cover the domain, hoping to discover in the 
factorial analysis the nature of the underlying order It is this latter appli¬ 
cation of the factorial methods that is sometimes referred to as an attempt 
to Lift ourselves by our own boot straps, because the underlying order m a 
domain can be discovered without first postulating it m the form of a 
hypothesis This is probably the characteristic of factor analysis that gives 
it some interest as a general scientific method 

Factor analysis is not restricted by assumptions regarding the nature of 
the factors, whether they be physiological or social, elemental or complex, 
correlated or uncorrelated For example, some of the factors may tuin out 
to be defined by endocrinological effects Others may be defined m bio¬ 
chemical or biophysical parameters of the body fluids or of the central nerv¬ 
ous system. Other factors may be defined by neurological or vascular rela¬ 
tions in some anatomical locus, still other factors may involve parameters 
in the dynamics of the autonomic nervous system; still others may be de¬ 
fined m terms of experience and schooling. Factor analysis assumes that a 


variety of phenomena within a domain are related and that they are de¬ 
termined, at least in part, by a relatively small number of functional unities 
or factors The factors may be called by different names, such as “causes,” 
"faculties,” “parameters,” “functional unities,” “abilities,” or “independ¬ 
ent measurements.” The name for a factor depends on the context, on one’s 
philosophical preferences and manner of speech, and on how much one al¬ 
ready knows about the domain to be investigated The factors in psycho¬ 
logical investigations are not ordinarily to be thought of as elemental things 
which are present or absent, like heads or tails in the tossing of coins 

The exploratory nature of factor analysis is often not understood Fac¬ 
tor analysis has its principal usefulness at the border line of science. It is 
naturally superseded by rational formulations in terms of the science in¬ 
volved Factor analysis is useful, especially in those domains where basic 
and fiuitful concepts are essentially lacking and where crucial experiments 
have been difficult to conceive. The new methods have a humble role They 
enable us to make only the crudest first map of a new domain. But if we 
have scientific intuition and sufficient ingenuity, the rough factorial map 
of a new domain will enable us to proceed beyond the exploratory factorial 
stage to the more direct forms of psychological experimentation in the 
laboratory 

In a domain where fundamental and fruitful concepts ai e already well 
formulated and tested, it would be absurd to use the factorial methods ex¬ 
cept for didactic purposes to illustrate factorial logic In such situations 
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there are available more direct methods of investigating lival hypotheses 
In the relatively young sciences and m the new domains of the oldei scien¬ 
ces, the factorial experiments will be useful It seems quite likely that the 
new methods will be applied with profit in the field of meteorology, but it 
is not likely that they will ever be used in classical mechanics 

In factorial investigations of mentality we proceed on the assumption 
that mind is structured somehow, that mind is not a patternless mosaic of 
an infinite number of elements without functional groupings The extieme, 
opposite view would be to hold that mind has no structure at all. In the 
interpretation of mind, we assume that mental phenomena can be identified in 
terms of distinguishable functions, which do not all participate equally in every¬ 
thing that mind does It is these functional unities that we are looking for 
with the aid of factorial methods It is our scientific faith that such dis¬ 
tinguishable mental functions can be identified and that they will bo veri¬ 
fied in diffeient types of experimental study No assumption is made about 
the nature of these functions, whether they aie native 01 acquiied 01 wheth¬ 
er they have a cortical locus 

In older to illustrate the method, let us consider a set of gymnastic stunts 
that might be given to a group of several bundled boys of comparable age 
A factor analysis staits with a table of intercorrelations of the vanables 
If there were twenty different stunts, we should have a square 20 X 20 table 
showing the correlation of every performance with every other performance. 
Our question now is to determine whether these relations can bo compre¬ 
hended in terms of some underlying order, which is simpler than the whole 
table of several hundred experimentally determined coefficients of correla¬ 
tion Let us suppose that some of the stunts require pnncipally sticngtli of 
the right arm, that others require pnncipally a good sense of balance, that 
still otheis requiie speed of bodily movement Sevcial tests that require 
good sense of balance might not require arm strength, while those which 
require a strong arm might require very little bodily balance. Wo might then 
find that the correlations can be comprehended in teims of a small number 
of functional unities, such as sense of balance, arm strength, or speed of 
bodily movement Each of the gymnastic tests might require one or several 
of these functional unities, but it is not likely that every test will lequire 
every one of the functional unities that are represented by the whole set of 
gymnastic tests A factorial analysis would reveal these functional unities, 
and we would say that each of them is a primary factor in the battery of 
tests Now, if we should take any one of these functional unities, such as 
sense of balance, and represent it in a new set of twenty tests of great variety 
which all required bodily balance, we might find that there are really several 
primary factors involved in this domain. For example, there might con¬ 
ceivably be a separate balancing factor for each of the semicircular canals, 
or there might be some other breakdown of the balancing factors that would 
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be revealed m an extensive study of balancing tests * A new set of more 
refined piimaiy factors might be found withm the domain of bodily balance 
This process might continue with the factorial investigation of more and 
moie restricted domains, as long as the functional unities continued to be 
difficult to conceive in dnect expei imenlation Eventually, the factonal 
methods, which aie essentially exploratoiy, would yield to the refoimula- 
tion of a problem in teims of the fundamental xational constructs of the 
science involved It is not unlikely that factorial analyses will point the 
way m the woik of inventing significant and fundamental scientific concepts 

Let us consider, next, an example in the sensory and perceptual fields 
Let us start with a set of twenty perceptual tests involving several of the 
modalities Some of the tests might requne visual acuity, otheis would re- 
qune keen discrimination of rhythm, still others might requiie speed of pei- 
ception Each of the perceptual tests might involve one or more of these 
functional unities, but few would lequire all these functions Some of the 
tests, for example, might, not depend on visual acuity In this simple case 
we should not be surpnsed to find factonally the primary functional unities 
that aie obvious at the stait 

If we turn to the moie central functions that aie involved in the intellec¬ 
tual and temperamental differences among people, it seems leasonable to 
suppose that heie also we may expect to find functional unities that will 
some day be as obvious as the sensory and perceptual unities are obvious 
to us now 

Oui woik in the factorial study of the human mind rests on the assump¬ 
tion that mind lepresents a dynamical system which can eventually be 
undeistood in teims of a finite numbei of paiameters We have assumed, 
further, that all these paiameters, or groups of paiameteis, are not in¬ 
volved in the individual differences of every kind of mental task Just as 
we take it foi granted that the individual differences m visual acuity are 
not involved m pitch discumination, so we assume that m intellectual tasks 
some mental or cortical functions are not involved m eveiy task This is 
the principle of "simple structure” or "simple configuration” m the under¬ 
lying ordei for any given set of atlnbutes 

Observation and educational experience lend plausibility to the concep¬ 
tion that the mental abilities are determined by a great multiplicity of 
causes 01 determiners and that these determiners are more 01 less structured 
01 linked in groups This multiplicity of determines can be thought of as 
a field of elements in which all aie not equally closely linked. Some elements 
may be quite independent in their actions, while otheis may be rathei closely 
associated The factois aie piobably functional groupings, and it would 
be a distortion to assume that they must be elemental We know precious 

Several such factors might appear m the place of specific variance and uninterpreted 
common-factor variance in the earlier studies 
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little about the determiners of human talent and temperament, and we 
should not impose upon our thinking an unnecessarily ngid causal frame 
If wc grant that men are not all equal m intellectual endowment and in 
temperament and if we have the faith that this domain can be. investigated 
as science, then we must make the plausible and inevitable assumption that, 
individual diffeiences among men can be conceived in teims of a finite num¬ 
ber of tiaits, parameters, or factors Rome of the faetois may be found to 
be anatomically determined, otheis will be physiological; while others will be 
defined, at first, in exponential, educational, and social teims As scientists, 
we must believe that a set of categones can be found for tho undei standing 
of mentality, which have, by their simplicity, a pnor claim on our concep¬ 
tual formulations. 

Factor analysis and empirical prediction 

Factor analysis involves a number of well-known statistical piocedures, 
and it is only natural for the student to begin his study of factoi tlieoiy with 
a statistical point of view In some lespocts this point of view is legitimate 
and useful, but there are some fundamental diffeiences between the objec¬ 
tives of factor analysis and the customaiy objectives in statistical woik 
that might as well be made explicit at the stait Some of the contiovcisial 
questions in factor theory have their souicc in misunderstandings about the 
objectives for which factor theory was developed 

Many statistical pioblems take a form m which a certain numbei of in¬ 
dices are available for each membra of a statistical population, and it is dc- 
sned to predict some new index in teims of the known or given indices fm 
each member of the population. A familial example in psychological woik 
is the piediction of student scholarship in terms of indices that aie available 
at the time of college entiance Piediction problems of this type aie un¬ 
solved by wutmg a iegression equation, in which the dependent, variable 
which is to be predicted is expiessed as a linear function of the independent 
variables The weight given to each independent vauable is called its “10- 
gression coefficient,” and the weights are so deteimincd as to minimize the 
sum of the squares of the lesidual enors The multiple eonelation coeffi¬ 
cient is the correlation between the piedicted values of the dependent viui- 
able and the actually observed values When the residual enois aie small, 
the multiple correlation is high Factor analysis diffeis fiom these .statisti¬ 
cal problems in that there is no distinction between independent and 
dependent variables In factor analysis one does not select some one vnn- 
able which is to be predicted cu determined by the other variable's All the 
variables in factoi analysis are tieated alike m tins sense Whcnevcx the 
investigator pivots his attention on one of the given vanables which is cen¬ 
tral in importance and which is to be piedicted by a set of independent 
vanables, he is not talking about a factor problem lie is then talking 



60 MULTIPLE-FACTOR analysis 

about a customary statistical problem, invoking a regression equation and 
multiple correlation. If be looks upon the whole set of variables as repre¬ 
senting a domain that is to be explored and if bis object is to discover wheth¬ 
er there is some underlying order among these variables, then he is talking 
about a factor problem Of course, it is possible to treat the same set of 
data successively by the two points of view In the first case we aie con¬ 
cerned about the prediction of one of the variables from the others, while 
m the second ease we try to discover some underlying older in the whole 


system of traits . 

It might be noted, in passing, that the distinction between independent 

and dependent variables is not intrinsic m the phenomena themselves. The 
differentiation between independent and dependent variables reflects only 
the attitude or purpose of the investigator What is an independent vari¬ 
able for one investigator can be a dependent one for another investigator 
working with the same data. Sometimes an author reveals his bias un¬ 
wittingly m the way he plots diagrams What he plots as a base line is 
likely to be his independent variable. What he plots as ordinates is likely 
to be his dependent variable. In this way we may show that a depression 
is caused by unemployment or that unemployment is caused by a de¬ 


pression 

In the factor problem as well as in the statistical prediction problem, one 
is concerned about residuals and how to minimize them This is piacticaliy 
universal m problems that involve observational data In the statistical 
prediction problem, one’s attention is centered on the one variable which 
is to be piedicted and on how to minimize the residual errors m that pre¬ 
diction, In the factor problem, one is concerned about how to account for 
the observed correlations among all the variables in terms of the smallest 
number of factors and with the smallest possible residual errors. Here we 
have the practical problem that, as we postulate more and more factors to 
account for the observed correlations, the residuals get smaller and smaller. 
Hence, every factor problem must deal with the practical question of de¬ 
termining when the addition of a new factor causes only a negligible reduc¬ 
tion in the residuals The mathematical theory of this problem has not yet 
been adequately solved, but there have been developed practical methods 
of dealing with it. There seems to be some question as to whether the na¬ 
ture of the problem is such that a rigorous solution exists, but that is for 
later discussion 

There are problems in which the distinction between multiple correla¬ 
tion and factor theory is not sharply drawn, and we shall describe one ex¬ 
ample here If a set of twenty independent variables is to be used for the 
prediction of a certain criterion, one’s first impulse might be to write a re¬ 
gression equation of twenty terms in which each of the twenty given vari¬ 
ables is weighted so as to give the best possible prediction of the dependent 
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vanable. In many situations such a solution would give 1 egression coeffi¬ 
cients which are so grossly in erroi that then absurdity can be seen by in¬ 
spection. This result happens if the supposedly independent vanables are 
not linearly independent If the intercorrelations of those twenty vanables 
can be accounted for in terms of, say, six factors, then we should be justi¬ 
fied m using only six terms in the regression equation Any additional 
terms m that equation would be likely to give absurd results, since they 
would imply that we rely on more factors than aie really present in the sup¬ 
posedly independent variables The intercoi 1 elations of the twenty van- 
ables might then be analyzed factorially in ordei to determine six gioups, 
each of which would give an average or composite These six composites 
could then be weighted in a regression equation of six terms for as many in¬ 
dependent variables Such a pioblem is, however, only a prediction prob¬ 
lem, and the prediction might be accomplished as well by different group¬ 
ings of the twenty given variables or indices In the moic fundamental 
factoiial problem the object is to discover whether the vanables can be 
made to exhibit some underlying order that may throw light on the proc¬ 
esses that pioduce the individual diffei cnees shown in all the variables 
In statistical prediction, the independent vanables are given, and the 
problem is to predict some one dependent variable The weights that are 
given to the independent vanables do not ordinarily have any dnect physi¬ 
cal interpretation, because they depend on the number of given vai mbles, 
their dispersions, and their corielations among themselves and with the 
variable that is to be piedicted. The weights or regicssion coefficients 
aie ordinarily treated meiely as numencal coefficients which happen to 
maximize the multiple correlation Then use is justified, not because of the 
meaning that can be attributed to their numencal values, but lather to the 
mere fact that a prediction can be made with the minimum lesidual, 

In factor analysis, all the given variables are treated as co-ordinate as 
regards independence or dependence The object of a factor problem is to 
account for the tests, or their intercori elations, m tcims of a small numboi 
of derived variables, the smallest possible number that is consistent with 
acceptable residual enors The weights that aie given to these derived 
variables are given more direct physical intei pretation than is usually the 
case with regression coefficients The derived vanables are of scientific in¬ 
terest only m so far as they ropiesent piocesses or paiametcis that involve 
the fundamental concepts of the science involved Whereas the statistical 
prediction problem demands meiely that a good piedietion shall be made, 
the factorial pioblem demands that them shall be a meaningful mteipreta- 
tion of the small number of derived variables in tcims of which the whole 
set of given variables can be comprehended 

Since it is the derived variables, or factors, that are the objective of a 
factorial analysis, one might regard them as dependent vanables. This m- 
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terpretation would agree with the usual meaning of the terms, independent” 
and “dependent,” if the factors had previously been isolated and if we had 
started out to determine which tests constitute the best appraisal of a given 
factor Such a problem would be similar to the familiar problem in test con¬ 
struction when the object is to determine the relative validities of several 
tests for a given cnteiion In the exploratory type of pi oblem for which the 
factorial methods were developed, the object is to discover the underlying 
order in a system of variables and to identify their nature In this use of the 
factorial methods there is no distinction between independent and depend¬ 
ent variables 

Psychological postulates and definitions 

The factorial methods have been developed primanly for the purpose of 
analyzing the relations of human traits These are defined as follows 

Definition 1 A trait is any attribute of an individual 

The factorial methods aie applicable also in the analysis of attributes of 
inanimate members of a group The members of a statistical population 
may be moments m time or regions m space 01 any other entities, each of 
which has a set of attnbutes This generalization will not be made ex¬ 
plicitly, but it is implied in the following chapteis Since the methods have 
been developed primarily with psychological categories in mmd, these will 
be explicitly discussed, even though the same methods axe applicable to 
problems which involve the attributes of inanimate members of a statistical 
group 

It is useful to distinguish between those tiaits which are descriptive of 
the individual as he appeals to others and those traits which are exemplified 
primarily in the things that he can do This distinction is involved m the 
definition of “ability.” 

Definition 2 An ability is a trait which is defined by what an individual 
can do 

This definition implies that there are as many abilities as there are enumer¬ 
able things that individuals can do Each ability is therefore objectively 
defined in terms of a specified task and of a specified method of appraising it 

Definition 3 The task, together with the method of appraising it, which 
defines an ability, is called a test. 

Definition 4 The numerical evaluation of a test performance is called a 

score 

It is implied m these definitions that an index of ability is covariant with 
the score m the test which defines the ability and that a true index of ability 
is covariant with the true score m the test 

It is desirable to develop the factorial methods m such a manner that they 
are independent of the assumption of normality of ability m any particular 
experimental population In the present theoretical development of the factorial 
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methods it will not be assumed that any of the distributions of ability aie nor¬ 
mal 

The application of the factorial methods m science rests on a fundamental 
postulate 

Postulate The standard scoies of all individuals in an unlimited nunibei of 
abilities can be expiessed, m fti st approximation, as linear functions of 
their standai d scores in a limited numbei of abilities 
The correlation between the tiue scores in two tests will be lefened to us 
the correlation between the two abilities which arc defined by the tests In 
statistical work it is customary to refer to two variables as “independent” 
when their con elation is zero The tenn independence will be used with three 
diffeient meanings They will be designated by appropnate adjectives un¬ 
less the context makes the designation unnecessary 
Definition 5. A set of n abilities are linearly independent if the ? ank of 
the matrix of their true mtercorrelations is n 
Definition 6. Two abilities are statistically independent in a population 
if their correlation is zero m that population 
Definition 7. Two obsei vations are experimentally independent if they 
are experimentally distinct, so that one is not dei ivcdfiom the othoi by a 
constraint either of the experimental situation or of the computations 
In one sense, no two observations can evei be experimentally independ¬ 
ent The teun can be used only with reference to the state of knowledge at 
the time that the obscivations aie made. 

It is clarifying to interpret gcometucally the lelations of abilities In 
such a context, two abilities that aie uncorielated in a population will bo 
called orthogonal in that population. Two abilities that are con elated m a 
population will be called oblique in that population 

There is special interest m the limited number of abilities in terms of 
which all other abilities oan be defined, since these aie landmarks in terms 
of which all abilities can be comprehended. 

The reduction of raw scores to standard scores 

A factor analysis starts, ordinarily, with a table of mtei con elations be¬ 
tween the variables that are to be analyzed These con elations are com¬ 
puted from the individual test scores for the experimental population It is 
of interest, therefore, to consider the reduction of the law scoies to denved 
scores, such as standard scores, and the eifcct of such reductions on the cor- 
i elation coefficients and on the resulting faetoual structure 

It will be useful to recall that the coefficient of eon elation between two 
variables * and y is unaffected if an aibitrary constant is added to the ads 
and if some arbitrary constant is added to the y’ a Fui thennore, the coef¬ 
ficient of correlation between the two variables is unaffected if the ads are 
multiplied by some arbitiaiy multipliei and if the y’s aie similarly subjected 
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to some arbitrary stretching factor These properties of the correlation coef¬ 
ficient can be summaiized in the statement that the correlation coefficient 
for two variables x and y is the same as the correlation between any linear 
functions of x and y Stated m symbols, we have 

( 1 ) T xy ~ Y(ax-\-b){cy+d) y 

where u f 6, c, d y axe arbitrary constants 

Two ways of reducing the given raw scores to standard form will be con¬ 
sidered here They give numerically different values for the standard scores 
and different mtercorrelations, 

Case 1, assuming that the distribution of ability is not necessarily Gaussian 
Let the given distribution of raw scores be represented by the skewed fre¬ 
quency curve at A in Figure 1 , with mean at uq, which is the mean of the 
raw scores in test j Let A,, denote the raw score of individual i in test j , 
The frequency distribution A of the raw scores X„ will be subjected to two 
changes, namely, (1) a translation to the position 13 without change in 
shape, so that the new mean will be at the origin 0, and (2) a stretching fac¬ 
tor, by which the distribution B is changed to the form C, which has unit 
standard deviation and mean at the ongin. The frequency distribution A 
of the raw scores X,, is changed by the translation to the frequency distribu¬ 
tion of scores (X,, — mf), as shown at B. This distribution is changed by 
the stretching factor to the distribution of standard scores, 



as shown at C . The translation is over the distance m 3 , and the stretching 
factor is l/o-,. The new distribution at C has its mean at the origin, and it 
has unit standaicl deviation The shape of the distribution C is exactly the 
same as the shape of the distribution at A as regaids degree of skewness 
The conelation of any variable with the raw scores X„ is the same as the 
correlation with the corresponding standard scores s Jt The important fea¬ 
ture of this reduction to standard scores is that the degree of skewness is 
unaltered and the correlation coefficients are unaltered 

Case 2, assuming that the distribution of true scores %n ability j is Gaussian' 
If the investigator wants to make this assumption, the reduction of the raw 
scoies X„ to standard scores involves an additional change in the frequency 
distribution, namely, to the normal distribution form The numerical values 
of the standard scores are not the same as in the first case, and the correla¬ 
tion coefficients between the variables are not the same The procedure is 
simply to regard the raw scoies as placing the individuals in rank order ac¬ 
cording to the ability represented by the test If the percentile rank of each 
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individual is determined from his raw score, the corresponding normalized 
standard score can be found dneetly and without furthei computation from 
the Kelley-Wood tables of the probability curve or from similar tables. This 
reduction to normalized standard scores gives a distribution that is normal 
with mean at the origin and with unit standard deviation The only func¬ 
tion served by the raw scores m this case is to ai range the individuals in 
rank older, and no attention is then given to the actual numencal values of 
the scores beyond their rank order 

These two ways of leducmg the raw scores to standard form raise the 
question as to which of the two ways should be adopted Psychological con¬ 
siderations entei into this choice, but fortunately the factorial lesults do not 
seem to be senously affected m those problems which are otherwise determi¬ 
nate In dealing with the raw scores on psychological tests, it must be rec¬ 
ognized that the scoring methods are, on the whole, quite arbitiary. So 
many points are allowed for this, and so many points aie deducted for 
something else The resulting raw scores in tests are as arbitrary as the 
scoring formulae adopted by the intuitions of the investigator The editorial 
composition of a paper-and-pencil test also has an effect on this problem If 
a large number of relatively easy tasks is included, the raw scores will rise. 
The dispersion of the scores and the shape of the distribution aie maikedly 
affected by the more or less arbitrary time limits that aie adopted for a 
test If the distribution is positively skewed, it can ordinallly be made to 
approach the normal shape by merely increasing the time limit foi the test. 
It takes very little experience in test construction to realize that the dis¬ 
tribution of raw scores has relatively little fundamental significance, and 
consequently it seems proper to regaid them as essentially not much more 
than a set of indices that place the subjects m rank order according to their 
ability to do the kind of task that is involved in the test If the intuitions 
of the psychologist in assembling a new test and in adopting a scoring for¬ 
mula are not too seriously in error, he will obtain an approximate rank oidor 
of his subjects that will remain essentially unaltered with reasonable shifts 
in editorial composition and in some changes of scoring formulae, even 
though the raw scores and the shape of their distribution might be subject 
to considerable alteration by such changes 

If the factorial methods were seriously dependent on the mvestigatoi’s 
luck in hitting some sort of “right” scoring formula and time limit for each 
test, the new methods would not be able to produce meaningful results 
The factorial methods are sufficiently powerful that one can take consider¬ 
able liberties with the raw scores without seriously affecting the results If 
we take a factor analysis in which several fundamental and meaningful fac¬ 
tors have been clearly identified, it would be instructive to subject the fac¬ 
tor methods to a severe test by radical changes m the original raw scores 
and then make a new factor analysis with the altered scores We might 
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change the scores in one test by writing their reciprocals instead, the scores 
in another test might be altered by writing their square roots, the scores for 
another test might be altered by wilting their logarithms, still another test 
might be altered by using the squaies of the original scores instead of the 
scores themselves; and so on with a new monotonic function for each test 
The rank orders of the subjects m a test would be either the same or com¬ 
pletely reversed We might use case 1 in reducing the scoies to standaul 
foim so as to retain for the eonelations all the distorted distribution shapes 
The correlation coefficients to be analyzed would then be markedly diffciont 
from those used in the onginal analysis, but it seems quite likely that the 
same basic factors would be identified This demonstiation has not been 
made, * but it probably would be successful m showing the powei of the 
factorial methods in isolating the undei lying order among the test vanables 
and the basic factors that determine the individual differences 

* Since this chapter was written, the demonstiation has been made, and it is desenbed 
in chap xu. 



CHAPTER II 

FUNDAMENTAL EQUATIONS 


Reference abilities 

In a multiple correlation pioblem one writes a regiession equation to ex¬ 
press a dependent variable as a linear combination of a set of independent 
variables Applied to psychological tests, the regression equation i& an at¬ 
tempt to predict a critenon—the dependent variable—as a lmeai combina¬ 
tion. of a set of tests—the independent vanables The observation equation 
which is the starting-point for multiple-factor theory is analogous to the 
familiar regression equation in ceitain respects. The scoie in a test is ex¬ 
pressed as a linear function of the scores in a set of postulated refeience abil¬ 
ities. Here the standard score s Jt of individual i m test j conesponds to the 
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dependent variable which is to be accounted for, but in the factoi problem 
all the individual test scores aie dependent variables, m this sense, that are 
to be accounted for. 

If an experimental population of N individuals has been given a set of 
re tests, the resulting tabulation of scores will be an nXN -score matnx B 
whose elements are the standard scores s 3 „ where the subsonpt j icfeis to 
the test and t refers to the individual (see Figure 1) If the test battery has a 
airly large number of tests, re, and if these covci a fairly restarted range, 
one nught attempt to account for the scoies s„ in terms of a smaller number' 
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scorns of mdmduT dajd SC ° ie , of L indlvidual » ™ test j, the s’s are standard 
scoies individual i in each of the unconelated reference abilities, and the 
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e’s are the weights assigned to the standard scores in the reference abilities 
for the determination of the observed standard scoie s,,. The equation is 
written to repiesent q terms m the right-hand member, and it is hoped that 
this number of reference abilities will be relatively small compared with the 
number of tests, n, in the whole battery 

The psychological interpretation of equation (1) is that a subject’s per¬ 
formance, s,„ on a test is detei mined in part by the abilities that are called 
for by the test and m pait by the degrees to which the subject possesses 
these abilities It should be noted that the x’s m equation (1) contain the 
subscript i but notj Hence the x’s aie descuptive of the individuals but not 
of the tests The c’s in the same equation have subscript j but not i Hence 
the c’s are descriptive of the tests but not of the individual subjects The 
first term repiesents the contribution of the first reference ability to the 
score, the second term represents the contribution of the second refeienee 
ability, and so on, for q reference abilities If the fiist refeienee ability is in¬ 
volved in the test j, then the coefficient c 3l will be large and positive If 
the individual i possesses this reference ability to a marked extent, then his 
standard score m that ability, x u , will be large and positive The pioduet 
will be an appreciable contribution toward a good scoie in the test If the 
second lefercnce ability is not involved in test j, then the coefficient c, 2 will 
be zero, and if the individual i has this second refeienee ability to a marked 
degree, his standaid score x 2l will be large and positive, but it will not con¬ 
tribute toward the score m the test because the test does not involve this 
second ability For this individual subject and test, the second teim van¬ 
ishes Similar interpretation can be made for high and low test coefficients 
and for high and low standard scores in the reference abilities. 

If the standard sco?'es of N individuals m n abilities are expressed as linear 
functions of their scores m r linearly independent abilities, where r<n, then 
the r abilities will be called reference abilities. 

It will be shown that if a battery of tests can be desezibed with reference to 
r orthogonal abilities, there exists an infinite number of sets of r orthogonal 
abilities in terms of which the description can be made with equal accui acy 
An arbiti ary set of r orthogonal abilities may be chosen foi purposes of de¬ 
scription These are the statistically independent or orthogonal leference 
abilities If a battery of tests can be desciibed in terms of r orthogonal refer¬ 
ence abilities, the tests can also be described by a set of r oblique reference 
abilities It is not necessary that a reference ability be represented by a 
test in which it is involved exclusively While each of the tests used m ex¬ 
perimental work defines an ability, it may happen that the refeienee abil¬ 
ities in terms of which tests and individuals are described are not represented 
by actual tests but by linear combinations of several tests A linear com¬ 
bination of tests may be thought of as a composite test. 

It should be noted that, even if each individual can be described in terms 
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of a limited number of independent reference abilities, it is still possible for 
every person to be different from every other person in the world Each 
peison might be described m teims of his standard scores m a limited num¬ 
ber of independent abilities The number of permutations of these scoies 
would probably be sufficient to guarantee the retention of individualities 

With a limited number of abilities not only does this formulation allow 
every peison to be different from every other peison, but it also allows the 
widest possible differences between several individuals who attain the same 
objective peiformance in a test This may be readily seen by considering a 
hypothetical example Assume that a test calls for two abilities, such as 
ability in abstraction and ability in the manipulation of numbers Several 
individuals try the test and attain the same score One of them may possess 
a high degree of ability in making the abstractions involved in the test, but 
he may be slow in numencal manipulation Another may be slow in foimu- 
latmg the abstract part of the problem, but he may make up for this de¬ 
ficiency by superior numencal speed. The objective lesult might be the 
same The purpose of factor analysis is to obtain a quantitative description 
of each fundamental ability in each individual by means of tasks that le- 
quue these abilities in different amounts Smcc eveiy task is probably com¬ 
posite in the abilities lequired, it is necessary to make the appiaisal of the 
abilities of individuals by analytical methods This is exactly the object of 
the multiple-factoi methods as applied to the problem of clescnbmg the 
abilities of people 

Factor analysis is reminiscent of faculty psychology It is ti uc that the 
object of factor analysis is to discover the mental faculties But the severe 
restrictions that are imposed by the logic of factor analysis make it an aidu- 
ous task to isolate each new mental faculty, because it is necessary to piove 
that it is called for by the experimental observations Factoi analysis does 
not allow that a new faculty be added as soon as a new name can be found for 
the things that people can do In order to piove that reasoning and absti ac¬ 
tion are two different faculties, foi example, it will be necessary to show that 
the tasks which call foi such activities really do involve two factois, and not 
one 

In the psychology of the future it may be found useful to postulaf e a dif¬ 
ferent form of ideal construct for the description of mental endowment than 
the simple one that is implied in equation (1) The ideal constructs of the 
future may involve elements with location in a space fiame with spatial, 
dynamic, and temporal constiamts analogous to the ideal constructs of ge¬ 
netics It would be unfortunate if some initial success with the analytical 
methods to be described here should lead us to commit ouiselves to them 
with such foice of habit as to disregard the development of entirely different 
constructs that may be indicated by improvements in measurement and by 
inconsistencies between theory and experiment 
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The factor matrix and the population matrix 

Equation (1) can be represented in 1 ectangulai form, as shown in Figure 1 
The factor matnx F contains the elements c, m , which aic descriptive of 
tests j in teims of refeience factors m The population matrix P contains 
the elements x mt , which are descriptive of individuals i in terms of inference 
factors m The cross product of a row of F and a column of P is represented 
in equation (1). If the factor matnx F is multiplied by tho population ma- 
tnx P, the product is the scoie matnx >S with elements s„, as shown in equa¬ 
tion (1) and in Figwe 1. The obseivation equation (1) can be written m 
the matnx form 

(2) S = FP , 
oi m the summational notation 

Q 

(3) Sji — ^22 c jm x vlt , 

»n= 1 

oi in the leetangulai fonn of Fig me 1. It will be assumed m mu use of 
this equation that the lefeicnce abilities lepiesented by columns of F aic 
oithogonal In a latci chapter the equation will be gcnciaimed fm the case 
of oblique oi correlated refeience factois 

This observation equation is tho stai ting-point fm the development of 
factonal theoiy It should be noted that all the c’s and all the re’s aic 
unknown and that we know only the individual scores s 3l in each test, at 
the start of a factonal analysis 

Interpretation of the test coefficients 

The entries c, m of the factoi matrix F desenbe the tests j in teims of the 
factois to, and they aie called the test coefficients oi facto) loadings Their 
inteipietation can be found in terms of the variance of the tests 

Since Sji and x m% are standaid scores, then sums foi the expemnental 
population are identically zeio Then, by (3), 

N N q 

W 2w S n = 22 = 0 ’ 

1 = 1 i =» 1 ?n 1 


and 


N 

3Cm\ — 0 

i = l 


(5) 
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The standard deviations of and of x mt aie both unity by definition. 


Hence 


(6) 

si*- 1 - 

i = l 

and 


(V) 

1 N 

iy T i = 1 
jV ^ 


The square of the standard score, is 

Q Q Q 

(8) Sji = Cj 1n X 1nt I (l7l Af ), 

m= 1 M=1 m= 1 

wheie m and M aie subscripts denoting the lefcience factors and where 
in ^ M Summing the squares of the standard scores over the experimen¬ 
tal population and dividing by N, we have 

1 ^, 1 ^. q ^ ^ ^ w <1 

(9) a = ~r "y', s jt ~ Jr ^ ^ c jn‘ x "i t ^ CjmXm 1 + Jr ^', 

t — 1 1=1 m=l M= 1 t = 1 vi = 1 

Rearranging the summations, we have 


(fo) <q 2 = 2 Cj “ 6 ' jm h S + = 1 

77L — 1 iV/ — 1 1=1 771=1 1=1 

But the correlation between each pair of orthogonal lefeience abilities m 
and M vanishes by definition, so that 


( 11 ) 


? mM 



2=1 


mi'&M 1 


0, 


and the z’s are standaid scoies by definition, so that they have unit standaid 
deviation Hence, 

(12) ff » = = 1 . 

1 = 1 

so that 


Q 



m— 1 
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This equation can be stated m the theorem that the sum of the squares of the 
lest coefficients of a test for all orthogonal factors is equal to unity 

It should be recalled that this theorem concerning the test coefficients 
assumes a set of uncorielated leference abilities Each test is hei e repi esprit - 
ed as having unit variance, since the test scoies aie assumed to be i educed 
to standaid form The squaie of each test coefficient,, r? jm , is that part of 
the total vanance of test j which is attributable to the lcfeicnce ability m 
The sum of these vananees is the total unit variance of the test 

The entiles in a factor matrix aie sometimes called factor loadings. The 
square of any cell entry m F, such as a] m , h'f, c) ]: uf represents the fi actional 
part of the total variance of a test attributable to a given factoi We have, 
then, the theorem that each factor loading or test coefficient c jm for orthogonal 
factors is the square root of the variance of test j attributable to the factoi m 

Common factors and unique factors 

The squaie of each test coefficient c jm in Figuie 1 can be mterpioted as 
the fractional part of the variance of test j which is attributable to the icfei- 
ence ability or factor m Psychological knowledge about tests enables us 
to postulate different lands of factois m F Some of the factois lepresent, 
abilities that aie involved m two 01 moie tests of a battery. These aie called 
common factors Anothei kind of factor is an ability which is involved in 
only a single test of a battery These are called specific factoi s These two 
types of factors can be lllustialcd with a psychological example In a but- 
teiy of tests there might be only one test whose some depends in part on 
writing speed Eoi that battery and for that test the ability to write fast 
would be a specific factor Peifoimanec in another test might, depend in 
part on acuity of hearing, and that ability might not be involved in any 
othci test of the batteiy. It would be a factor specific to one test in the bat¬ 
tery Other factois, such as word fluency 01 numbei facility or visualizing, 
might be m two or more tests of the batteiy so that they would be common 
factors 

The common factois and the specific factois aie thought of as bona fide 
abilities, whose isolation and description are of psychological interest But, 
in addition to these cognitive factors, it is well known that we must deal 
also with foituitous cnors in test scoies If we knew the true scores in a 
particular test for a given population, the addition of foituitous chance 
enois to the tiue scoies would increase the dispcisions so that the true acme 
vanance would be some fraction of the observed variance This on or part 
of the total vanance of each test should be lopiesenled by an error factor 
m the factor matrix F Since the fortuitous vanable enois in one test arc, 
in general, uncorrelated with the variable cnors in each otlioi test, wo must 
provide m the factor matiix F one onoi factor for each test of the batteiy. 

In Figure 2 we have represented six tests with three kinds of factois— 
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the common factors, the specific factors, and the error factors Sampling 
errors will be discussed m a later section In Figure 2 we have shown three 
common factors m the fiist three columns. The letters a , b, and e represent 
non-vanishing cell entries The blank cells lepresent zero entries The com¬ 
mon factors, I, II, III, are heie shown as if each common factor were pres¬ 
ent in each test. Later we shall considei the situation m which the common 
factors are not all present m each of the tests The next block of the factor 
matrix of Figure 2 shows eight specific factois. Two of them occur only in 
the first test We have labeled them i and 5 The third test is also shown 
with two specifics, and each of the other tests is represented with one spe¬ 
cific factor Each test is assigned a variable error factor e„ m the third block 
of the factoi matrix 
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The total variance of the first test in this fictitious example can be ex¬ 
pressed m the summation 


(14) (ah + ah + ah) + (&? 4 + &? 6 ) + eh = 1 , 


on the assumption that all the factors involved in the test can be divided 
into thiee types—common, specific, and erroi Only the common factors 
are shared with any of the othei tests m the batteiy Each of the other 
factors is unique foi one test 

The total variance of the test can be divided into two pai ts, namely, that 
pait which it shaies with other tests m a particular battery and that part 
which is unique We then have 


h] = communcthly of test j , 

m = 1 


(15) 
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where the a’s refer only to common factors and r is the number of common 
factors The notation hf will be used to denote that part of the total vari¬ 
ance of a test which is attubutable to the common factors 
Definition. The communality of a test is its comrnon-fadoi vaiiance 
The specific factois aie those abilities which are involved m only one test 
of a battery Then vaiiance can be written 

<1 

(16) b* m s= specificity of test j , 

Wl=" 1 

where the V s refer only to specific abilities. 

Definition The specificity of a test is that pail of its total vai iance which is 
attiibuiable to abilities that are unique foi the test m a given battery 
The specific factors and the variable enoi factor are all unique foi each 
particular test The fractional pai t of the total variance of a test which it 
does not share with any other tests of the batteiy can be wntten 

i 

(17) 2^?’" e ’ 3 “ u i = uniqueness of test .7 . 

Definition The uniqueness of a test is that part of its total vai iance which 
is not shared with any other tests m a battery 
It will be seen that the uniqueness is the complement of the communnlity, 
so that 

(18) h) + «* = 1 

A factoi that is specific foi a test in one batteiy may become a common 
factor when the test is moved into anothei battci y Auditoiy acuity might 
be specific in one battery, but it might become a common factoi if a test 
were moved into a new battery, whore two or more tests depended on audi¬ 
tory acuity 

AH unique factors of a test may be combined into a single unique fac¬ 
toi for each test As an example, the unique-factoi entry foi the first test 
m Figure 2 would bo 

^b'u + 6j c + ej, = «, , 

as is also shown m equation (17) It will be useful to ic-wntc the factoi 
matux F with two blocks instead of thiee These two blocks ai e the common 
actors and the unique factors This has been done 111 Figure 3 
The pattern of the non-vanishing enti ics m F should be noted We have 
shown the common-factor block in F as filled, whereas the unique-factor 
block is a diagonal pattern with zeios in the side entries This pattern foi- 



76 MULTIPLE-FACTOR ANALYSIS 

lows from the definition according to which each of the six factors occurs in 
only one of the six tests 

Definition' A factor matnx which represents the total unit variance of each 
test is called a complete factor matrix. 

The complete factorial matrix is denoted 1<\. The factorial matrices of 
Fiqures 2 and 3 are complete factorial matrices In factor analysis it rs use¬ 
ful to identify by a special name that part of the factorial matnx which rep¬ 
resents only the common factors The common-factor block is a matrix of 
ordei n'X.r for n tests and r common factois In Figure S this matrix is of 
order 6X3 with elements a jm 
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Factor Matrix F, 

Figure 3 

Definition A factor matnx which represents only the common-factor vari¬ 
ance of each test is called a reduced factorial matrix. 

The reduced factorial matnx is denoted F. 

Fiom this definition and equation (15) we have the theorem that the sum 
of the squares of each row j of the reduced factorial matrix F is the communahty 
h^ of test j 

If we postulate a unique faetoi for each test, as shown in Figure 3, the 
total numbei of linearly independent factors in F i is where ? is the 

number of common factors But this formulation of the pioblem involves 
moie factors than tests, a situation that would seem to make the problem 
insoluble Befoie a stable solution can be expected by any method of anal¬ 
ysis, the factor pioblem must be so foimulated that the number of observa¬ 
tions exceeds the number of independent parameters to be dcteimined, 

The intercorrelations 

The square table of intercorrelations of n tests is called the correlation 
matrix Such a matrix is illustrated m Figure f The con elation matrix is 
square, and it is symmetric because r }k — r kJ When the correlation matrix is 
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written with unity in the diagonal cells, it is called a complete correlation 
matrix, and it is denoted Ri The correlation matrix can be expressed in 
teims of the matrices that have been defined m pievious sections The cor¬ 
relation between thcstandaid scores s 3l and s*„ where the subscupts j and k 
both refer to tests, is 


(19) 


r,K 


1 




1=1 


since the standard scores have unit standard deviation 
foim 


(19a) 


r lk = 



i=i 


SjiS-iJc f 


If we write it in the 


we imply the multiplication of two score matrices, namely, s,, and s,* 
The score matrix with elements s } , is of order nXA 1 and it is denoted S The 
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scoie matrix with elements s lk is the transpose of S and is of oidei NXn 
Hence we have 

(20) R 1 = ± SS 1 , 
where the diagonal entries of R i are unity, i e , 

( 21 ) r„ = = 1 

1=1 

By (20) and (2) we have 

(22) fti = ~ ( I<\P){I<\Py 

Ri = if FiPP'F[ 


(23) 



7g multiple-factor analysis 

The product PP 'm (23) represents the cross products of rows of P, These 
can be written m the form 

N 

(' 21 ) PP J — 



r n 


n 


Factor Matrix F„ 

Figure 5 

and M These correlations are zcio, except the diagonal self-correlations, 
which are unity Hence 

(26) PP’ = NJ , 

where I is the identity matrix Substituting in (23), we have 

(27) R ! = FJf'l . 

This equation is fundamental in factor analysis It can be stated as a funda¬ 
mental theorem, namely, that the product of the complete factorial matrix by 
its transpose is the complete correlation matrix 

The complete factorial matnx Fi of Figure 3 can be expressed as a sum 
of the two matuces shown in Figure 5 The matrix F T is of order n by (w+r), 
where the unique factor loadings have been omitted The matrix F u is also 
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of order n by (n+r), where the common factor loadings have been omitted 
If we superimpose these two matnees, we get a matiix sum of the type shown 
in Figure 3 Hence we wnte 

(28) 1? r + F u = Fi . 

Substituting in (27), we have 

(29) Ri = (Fr + F u )(F t + F u y 

or 

(29a) Ei = (F r + F U )(F' 4- E,' ( ) 

This becomes 

(30) R\ = F r F’ r 4- FrFi + F u F'r 4- F U F U , 

in which F r and F u are both of order n by (n4-r) These foui separate prod¬ 
ucts in (30) are shown in rectangular notation in Figure 6 In older to sim¬ 
plify Figure 6, it has been paiticulanzed to the case of n = 4 and r= 2 The 
products /'V/'u and F U F(. both vanish Hence wo have 

(31) Ei = FrF’r 4- F u F( t 

The product F U F’ U is shown in i octangular notation in Figure 6 The piod- 
uct F U F’ U is a square matrix of order nXn with diagonal entries v? u . It is that 
part of the con elation matnx which shows the contribution of the unique 
factors to the con elation matnx Rx 

The pioduct F r F’ r m (31) is shown in rectangular notation in Figure G 
This product is that pait of the conelational matnx R x which is conlnbuted 
by the common factors It will be denoted E without subscnpt Note that 
R contains the xnteicorrelations of the tests and that the diagonal elements 
are the communahties instead of unity asinEj It is of significance that 
the unique factors contribute only to the diagonal cells m the correlation matrix, 
as illustrated m Figure 6 The matnx R will be called the reduced correlation 
matnx because its diagonal (dements have been depi cased fiom unity to the 
communahties 

The product F r F' n whore F r is of onlci n by («4-?’)i as lllustiated m Fig¬ 
ure 6, is the same as the pioduct FF', where F is a leduced factoi matrix 
of oidei nXr, as shown in Figure 7 We have, theiefoie, the important 
equation 

(32) 


FF' = E 




Figure 6 
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winch can be stated as the fundamental factor theorem The •product of the 
reduced factor matrix by its transpose is the reduced correlation matrix. This 
fundamental relation between F and R is shown in rectangular notation m 
Figure 7. 

The complete correlation matrix R i has now been divided into two paits, 
namely, R and R u , 

(33) R\ = R R u , 

where R u is a diagonal matrix This summation is shown in rectangular no¬ 
tation in Figure 8 Since each diagonal element uj of R„ shows the unique 
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Figure 8 

variance of test j, the complete correlation matrix Ri and the reduced corre¬ 
lation matrix R differ only in the diagonal cells The diagonals of R contain 
the communalities. 

If we regard the correlation coefficients as the given data and the factor 
matrix F as the desired objective m a factorial analysis, we have n(n — 1)/2 
experimentally given coefficients which must exceed the number of linearly 
independent parameters in F It will be seen that, by limiting ourselves to 
the common factors of F and the corresponding intercorrelations, the factor 
problem becomes determinate, even though we admit the existence of unique 
variance, which is known to exist in all psychological tests. This restriction 
of the factor problem to the common factors with no attempt to account for 
the total variance of each test is one of the turning-points m finding a de¬ 
terminate solution m factor analysis as a scientific method. 
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In general, the rank of the complete correlation matrix is equal to its 
order, namely, n Since the reduced correlation matrix R is the product 
FF' r it follows that the rank of R cannot exceed the rank of F . The reduced 
factor matrix F is of order nXr, where r <n Since the columns of F repre¬ 
sent linearly independent factors, the rank of F is equal to the number of 
common factors r Hence we have the important theorem: The number of 
common factors represented in a reduced correlation matrix is equal to the rank 
of the matrix. Experimentally obtained correlation coefficients are subject 
to sampling errors, which constitute a fortuitous element in each coefficient 
In the strict sense of the term, the rank of a con elation matrix for experi¬ 
mental data is always equal to its order, namely, n The scientific problem 
is to account for as much as possible of the relations between tests by the 
smallest possible number of common factors This leads necessarily to a 
problem of determining when the residuals are negligible for any given num¬ 
ber of common factors The computational problem is made more difficult 
because the communahties m the diagonal cells of R are unknown These 
diagonal elements must be estimated at the stait of an analysis in finding 
the factor matrix F for a given set of correlations in R 

In the mathematical sense, the numbei of linearly independent factors 
involved in the correlations between n tests is the rank of the correlation 
matrix, irrespective of what the diagonal elements may be In general, one 
would limit this reduction of correlations into factors by considering only 
correlation matrices whose diagonal elements do not destroy the Gramian 
properties of the correlation matrix In stating the theorem concerning the 
number of factors in a correlation matrix explicitly for the reduced corre¬ 
lation matrix, we have in mind the scientific problem of discoveung physi¬ 
cally identifiable factors as distinguished from numerical aitifacts which 
might be permissible as mathematical solutions but ai e not allowable as sci¬ 
entifically significant solutions in a factor problem 

The correlation matrix for psychological tests with unity in the diagonal 
cells has, in general, a rank that is equal to its order, namely, n Since we 
have for the complete correlation matrix the relation Ri=F 1 F(, we should 
expect n linearly independent factors in Fi But by psychological considera¬ 
tions we expect ( n-\-i) = q factors m F 1 Hence the columns of Fi are not 
linearly independent It will he shown that R lt which is of rank n and order 
n, can be factored into a factor matrix with as many factors as there are 
tests, but such a numerical solution is not ordinarily of much interest or 
value, since it lepresents a condensation of (w+r) factors into n linearly in¬ 
dependent columns of numbers m F\, which laxely correspond to the r 
linearly independent psychological common factors of the intercorrelations 
This numerical solution with as many factors as there are tests will be de¬ 
scribed in detail in a later section 
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The reliability coefficient 

The correlation between two administrations of the same test or between 
two tests that aie designed to be paiallel is called the lehability coeffi¬ 
cient of the test. An example is the con elation between the scoies m two 
spelling tests of one hundred woi ds each, when the two lists have been cliawn 
as random samples from a longer vocabulaiy. If the two tests aie truly 
parallel, they must involve the same abilities, common and specific, but the 
vanable chance errois in the two tests are independent. This situation is 
represented in Figure 0 

It is customaiy in psychological woilc to write the lehability coefficient 
in the diagonal cells of a coil elation matnx By the piesent analysis it is 
seen that the diagonal entries of R 1 are unity, while the diagonal entries of 
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R aie the communahtics h'f. The lelation between the icliability and the 
communality of a test may be shown by eonsidcnng in detail the faetonal 
matiix for a test j, a parallel test /, anothei test k, and its parallel test k’ 
The factorial matrix foi these foui tests is shown in Figure 9 

Let there be r common factors in the foui tests Let b , 1 be the specific 
variance m test j Since j and / aie paiallel tests, it is evident that they 
must requne the same common abilities and the same specific ability 
Hence b , is lccoided m the same column of F i in Figure 9 for both j and f. 
For the same leason, 6*. must be common to tests k and k', which are paral¬ 
lel But the vanable onors aie uneonelated by definition, even foi parallel 
tests. Hence F i shows a separate erroi factor foi each of the four tests 
The coirelation between the tests j and j' is the lehability coefficient of 
test j, and it will be denoted i n Since Ri — FFi, the lehability of j can be 
found m Figure 9 as the cross product of the fust two rows of the matrix. 
We then have 


r u = S a ’"‘ + b >> 

m—1 


(34) 
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and the reliability of test k is 

r 

(35) r k k ~ ai»+ . 

m = 1 

By (34) we have 

(36) r„ = h* + bj . 

Since the total variance of j is unity, we have 

(37) i'jj = 1 - e? 

This equation states the theorem: The reliability of a test is the complement 
of its error variance 
Since 


(38) 

cl-*-. 

s 

1 

T—l 

II 

d 

and 


(39) 


we have 


(40) 

h* = 1 - bj - ej 

or 


(41) 

W, = r„ - b ), 

and hence 


(42) 

h] ^ r„ . 


This inequality states the theorem The communality of a test is always small¬ 
er than the reliability except %n the limiting case where the specific factor is ab¬ 
sent, m which case the communality and the reliability are equal 

It is of interest to note that the factorial methods effect a separation be¬ 
tween the communality and the uniqueness of each test in a battery but 
that the uniqueness cannot be separated into its two principal parts— 
specificity and error—by factorial methods In order to estimate the specific 
variance of a test or its reliability coefficient, a separate experiment must be 
made with a repetition of the test or with parallel forms of it If the reli¬ 
ability coefficient is determined by independent experiment, the specific 
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variance can be estimated as the difference between the reliability and the 
communahty 

An object of psychological inquiry is to isolate an increasing number of 
abilities until the specific variance of each important test is reduced to a 
minimum. It is not likely that any single test will be completely described 
in terms of the factors which it has in common with those of one batteiy. 
In order to isolate all the abilities that are called for by a test, it will proba¬ 
bly be necessary to insert it in seveial test batteries in succession The spe¬ 
cific variance of a test should be regarded as a challenge; it is that pait of 
the total variance of a test which is unique in a particular battery, and hence 
its factorial composition is unknown. In order to test a hypothesis concern¬ 
ing the abilities which are involved in the specific vanance of a test, the test 
should be combined with others which involve the hypothetical abilities 
If the specific variance is reduced, the hypothesis is sustained 

For the next few years it will probably be more interesting to isolate new 
abilities than to reduce the specificity in particular tests Increased knowl¬ 
edge of the primary mental abilities will facilitate the type of experiment 
by which the specificities of particular tests may be reduced It will prob¬ 
ably be found that a considerable fi action of the total vanance of each test 
is attributable to factois of such limited social significance that the com¬ 
plete elimination of the specificity of each test will not be essential in the 
early stages of the scientific study of human abilities 

Summary of terminology and notation 

The terminology for the different parts of the vanance of a test is sum¬ 
marized as follows ■ 

Total vanance ~ 1 = h] + b'f -j- = hj + u) ; 

Reliability ~ r„ = h] + b] = 1 — - 

Communahty = h* = h) = 1 _ u * . 

Specificity = b] = b] , 

Uniqueness = u'f — l* -f e j , 

Error variance = e] — e* = 1 — Tjt . 

The notation for scores, matiices, and matrix elements is as follows 

N = number of individuals in expeiimental population, 
j, k = two subscripts for tests, 
s„ = standard score of individual i in test j, 
i = subscript for individual in experimental population, 
m, M = two subscripts for arbitrary oithogonal reference abilities or 
factors; 

Xm > = standard score of individual i in reference ability m, 
c 3m ~ test coefficient of test,; for the factor m, 
r = number of common factors in correlation matrix; 
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n -number of tests in the battery] 

=conelation between tests j and Is, 
r n reliability of test j, 

a )m = test coefficient of test j in the common factor % 

F i = complete factor matrix for total variance of each test; 

F =1 educed factor matrix for common-factor variance in each test, 
P = population matrix with elements x m , 

I), = specific factoi loading in test j, 
e j =enoi* factor loading in test j, 

u, = unique factoi loading in test j, 

S = score matrix with elements 8 , M 

<rj = total vauance of test j=l, 

4 = vauance of scores 4.=1; 

ft - complete correlation matrix with unity in diagonal cells , 

R = reduced coirelation matrix with communalities m diagonal cells 



CHAPTER III 
GEOMETRICAL MODELS 

Geometrical representation in factor analysis 

In understanding the lelation between the factoi matrix F and the eoire¬ 
lation matrix R, it will be helpful to consider a set of geometrical models 
The algebraic development of factor theoiy could be lepicsenled by scveial 
different geometrical analogies, but we have selected one that seems to be 
the simplest and most direct, and it will be used throughout this text In 
this chaptei we shall consider the geomelnoal repiesonlation of pioblems 
that involve only two common factois These pioblems can all be lepi esent- 
ed by two-dimensional diagrams In later chapters we shall extend the same 
kind of geometncal mteipretation to pioblems involving thice common 
factors which can be repiescnted by thiee-dimensional models The same 
geometrical mteipretations will then be extended to the ^-dimensional 
methods of multiple-factor analysis Most students find it veiy helpful to 
visualize the problems of factoi analysis by geometncal leprcsentahons 

Factorial structure 

In Figure 1 we have repiescnted a pioblom that involves eight tests with 
two common factors The couclational matiix R is shown at te m tlie figuie 
and the factor matiix F is shown at lb 

The factor matrix F will now be given a simple geometncal m lei pi elu¬ 
tion Let the entiles in each row lepiescnt the co-oidmates of a point in the 
diagiam at la m the figuie The fust iow gives the faetoi loadings 70 and 
.30 for test 1 These two numeneal values aie mtcipretcd as the eo-oidmafes 
of a point which is plotted in the diagiam la This point is the lei minus of 
the test vector 1 The test is now lcpresentcd by a test vector Each of the 
eight tests in the factor matrix F is represented m a similar mannoi in the 
diagram at la, and we then have as many test vectois in the diagiam as 
there are rows in the factor matiix F In addition, we have the two unit, 
reference vectors A and B, which lepiesent the factors that are shown by 
the columns of F The combination of the test vectois and the lcfncncc vectois 
for a test battery is called a factorial structure + The faetonal stiuctuie foi 
the present lllustiative example is shown m the diagiam at la 

In the factor matrix there aie foui tests, namely, 1, 3, 4, 7, which have 
non-vamshmg entiles in both columns of F These aie lepiesenled m the 
middle of the first quadiant of Figure la. Two of the tests—2 and 5-—hav<> 

* These definitions of structure and i elated concepts arc the same as in the wntci’s 
original publications on multiple-factor analysis (1931-36), 
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non-vanishing entries only m column A These two tests have corresponding 
test vectors that are colhnear with the reference vectoi A Two other tests 

_ Q a nd 8—have non-vanishing entries only in column B of F These are 

represented in the corresponding structuie by test vectors that are collineai 
with the reference vector B 

The geometrical representation of the test battery can be geneialized in 
the statement that each factor loading a, m in F is represented as a projection 
of the test vector j on the reference axis m. The first test vector has a projection 
of 70 on the reference vector A and a projection of .30 on the reference vector 
B, The second test vector has a projection of 90 on the reference vector A 
and zero on the reference vector B This means that the second test vectoi 
is orthogonal to the reference vector B If a factor loading n im in F is zero, 

n 


y 


k 


FiourtE 2 

the geometrical interpretation is that the corresponding test vectoi j is orthogonal 
to the reference vector m. 

Correlations as scalar products 

The intercorrelations can also be represented geometneally in a veiy use¬ 
ful way Each of the tests is represented in Figure la as a test vector It 
will be shown that the correlation between any two tests is the scalar prod¬ 
uct of their test vectors The scalar product of two vectois j and k is 
h,hic cos 4>ik, where h, and h k denote the lengths of the vectors and 4>jk is theii 
angular separation 

In Figure 2 we have represented two test vectors j and k, whose lengths 
are h, and h k , respectively. Adjacent to the figure is also the factor matrix 
for the two tests j and k in their own plane with orthogonal axes I and II 
The factor loadings, aj k and aji, are the projections of the test j on the two 
orthogonal axes, as shown in the figure, and a similar interpretation applies 
to the factor loadings, a k i and a k i of test k 

The correlation r Jk between the two tests can be determined by the prod- 
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uct of the factor matrix and its transpose The mtercoirelation is then the 
cross product of the two rows of the factor matnx, namely, 

(1) r 3 k = 

Dividing by h,h K , we obtain 


( 2 ) 

but 


1 }k _ fljl _ fflU , Oj. 012 

hjhk h, hk h, h k ' 


(3) 


h 


cos ^>ji, 


(4) 


T — COS tpk 1 , 
Ilk 


(5) g = am ^ , 

( 6 ) = sin <j>n , 


where <f>,i and are the angular separations between the test vectors j and 
k from the first reference axis We have, therefore, 

~ ^os $ii T $71 


The angle 4> ]k can be expressed as the difference between <p 3 i and <j>u, so 
that 

® 4>jk — 4>j i — <fni ; 

but the right-hand member of equation (7) is the expiession for the cosme 
of the difference between the angles 4 >,i and <f, kl Hence we have 


(9) Kft k = cos +* ’ 

or 

( 10 ) r, h = hjhk cos 4>, K 

The correlation between any two tests is equal to the scalar product of their test 
vectors 

An important fact about this theorem is its independence of the reference 
axes. It should be noted that equation (10) does not involve the reference 
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axes. Since the scalar product is determined only by the lengths of the vec¬ 
tors and by their angular separation, the scalar product of a pair of vectors is 
invariant under rotation of the reference frame. We can now make an inference 
which involves one of the central ideas m factorial theory, namely, that the 
correlation matrix determines the test vectors as to their lengths and angular re¬ 
lations, hut the correlation m.alnx does not define the location of the reference 
axes 

With the scalar-product interpretation of the conelation coefficient, it 
follows that a negative coefficient is always represented by an obtuse an¬ 
gular separation of the test vectors, the lengths h, and h k being taken as 
positive Also, with non-vamshing lengths of the veotois, a zero correlation 
is represented geometrically by a right angle A pair of tests vnth zero cor - 
relation have orthogonal test vectors 

If two vectors are each of unit length, then the numerical values of h, 
and h k are both unity, and their scalar product is the cosine of their angular 
separation But cos is also the projection of either unit vectoi on the 
other Hence we have the theorem that the scalar product of a pair of unit 
vectors is equal to the projection of either vector on the other 

If the entries m the correlation matrix aie all positive, as is the case with 
tests of the cognitive or intellective functions, then all pairs of test voctois 
are separated by acute angles. It follows that if all the cot relations me 
positive, the configuration of test vectors lies inside a cone of generating 
angle 

The configuration 

The correlation matrix of Figure le contains all the conelations for the 
eight-test battery. Since only two common factors aio involved in these cm- 
relations, the rank is 2, and it is possible to lepresent the test lelations by a 
set of eight test vectors in two dimensions The lengths of the test vectors 
are determined by the diagonal entries in the conelation matnx. Each diag¬ 
onal entry hf is the communality of a test j ,and the length of the coi i espond- 
mg test vector is h, The angular separations between the test vectors ai e de¬ 
termined by the side entries of the correlational matrix m such a way that 
the scalar products of all pairs of vectors ai e equal to the cori ospondmg coi- 
relations. The number of the dimensions that aie lequned to represent the 
given correlations is the same as the rank of R, and this is the same as the 
number of common factors in the test battery 

If we consider only the arrangement of the test vectors among them¬ 
selves as determined by the correlation coefficients and without regard to 
any reference frame, we call the ariangement or grouping a configw ation of 
test vectors The configuration of test vectors foi the present example is 
shown in Figure Id The configuration at Id is identical with that at la 
The only difference is that the configuration and the reference axes aie com- 
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bined to foim a structure at la, whereas the configuration alone is shown at 
Id without any reference frame The configui ation has been drawn inten¬ 
tionally m different positions at la and Id 

There is an exact relation between the test configuration at Id and the 
correlation matrix at le A model or diagram of the configuration can be 
constructed from the correlations, and the reverse is also possible, m that 
the con elations can be computed from the configuration We lose no infor¬ 
mation m going from the correlations to the configuration or vice versa be¬ 
cause they are merely two different ways of stating the same facts It should 
be noted especially that neither the correlations nor the configuration has any 
reference fi ame 

Just as there is a direct correspondence between the correlation matrix 
and the test configuration, neither of which involves a reference frame, so 
there is a direct correspondence between the factor matrix and the factorial 
structure, both of which do involve a reference frame The pairs have been 
placed adjacent to each other in Figure 1 to emphasize these relations The 
test structure cannot be defined until the reference axes have been located 
in the configuration, as shown m Figure la In the same manner, the factor 
matrix at lib cannot be defined except in terms of a paiticular set of refer¬ 
ence axes which are implied by the two columns of that matrix 

A factor problem usually starts with a table of intercorrelations, as shown 
m Figure le If it were detei mined that the rank of that matrix is 2, then we 
should know that the mtercorrelations of the eight tests are produced by two 
common factors even before we could have any idea what these two factors 
are We would also know that the configuration of test vectors would be con¬ 
tained within two dimensions, so that they could be represented as a plane 
diagram, as shown at Id When the correlational matrix is given, including 
the diagonal entries, then the configuration can be diawn It should be espe¬ 
cially noted that the correlation matrix and the test configuration have no 
reference axes 

The object of a factorial problem is to wiite a factor matnx F that will ie- 
produee the correlations R by the fundamental theorem FF' = R This prob¬ 
lem can be represented geometrically It consists in determining the rank 
of R, winch is also the number of dimensions in the test configuration, and 
the lank is also the numbei of columns of the factor matrix F The con¬ 
figuration can then be drawn uniquely A set of reference axes can then be 
inserted into the configuration to form the structure at la When the ref¬ 
erence axes have been inserted into the configuration, it is possible to de¬ 
termine the projection of each test vector on each one of the reference axes 
These projections constitute the cell entries a jm of the factor matrix F. It 
should be clear, then, that the factor matrix cannot be written until a set 
of reference axes has been inserted into the configuration which is defined 
by the mtercorrelations 
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Certain features of tlie test configuration can be inferied by meie inspec¬ 
tion of the correlation coefficients If all the correlations are positive or 
zero, we ean infer that all the test vectors are separated by acute angles 
The laigest angular separation would then be a right angle It follows that 
the whole test configuration must be contained within a cone of 45° gen¬ 
erating angle when the correlation matrix has no negative entries This is 
the situation with psychological tests of the cognitive or intellective func¬ 
tions, which seem never to show any significant negative correlations 

The arbitrary reference frame 

Since the object of factor analysis is to find a factor matrix F from a 
given correlation matrix Ei so that the fundamental factor theoiem FF' — It 
is satisfied, it is of interest to note that every correlation matnx can be fac¬ 
tored into a factor matrix In fact, the correlation matrix is, by definition, a 
product of two score matrices, as has already been shown in (20-n), 

(20-u) Ei = ~ SS' . 

Since the coi relation matrix is obtained computationally as a product of two 
matrices, it is evident that every conelation matnx can be factored. We 
have, therefoie, the theorem that for every coi relation matnx It there exists a 
corresponding factorial matnx F such that FF' = R 

We have seen that a given correlation matnx can be interpreted as show¬ 
ing the scalar products of pajrs of test vectors. We have also seen that the 
factor matrix can be inteiprcted as showing the projections of each test 
vector on each of the co-ordinate axes The numencal values of the piojco- 
tions of the test vectors are, of course, dependent on where we put the fac¬ 
torial reference frame Hence we have the important thcoi cm that an in¬ 
finite number of factor matrices F can be written which reproduce any given cor¬ 
relation matrix R. This theorem makes it evident that when wc have fac¬ 
tored a given correlation matrix R into a factor matrix F we do not have a 
unique solution Further restrictions must be imposed on the solution F, 
so as to make it m some sense unique before we can hope to interpret the 
test coefficients On this question there has been much controversy, which 
we shall have occasion to consider in a latei chapter. 

In order to demonstrate the arbitraiy location of the reference frame, we 
have drawn in Figure 8 several positions of the co-ordinate axes foi one test 
configuration The configuration of Figure Id is reproduced in tliroo differ¬ 
ent positions in Figure 8 as legaids the co-ordmate axes Below each dia¬ 
gram in the figure we have the corresponding factor matrix. Although the 
numencal values are different m these three factor matrices, they all repro¬ 
duce the same coi relations, namely, those of Figure le 

Since the correlation matnx R consists of scalar products of test vectois, 
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while the factoi matrix F shows the projections of the test vectors on the co¬ 
ordinate axes, we have several other theorems that are useful in under¬ 
standing the relations between these matnces, R and F . Consider the test 
vector k and the unit vector X m Figure 4■ The scalar product of these two 
vectors is h :Ji T cos cLt, which becomes h k cos 4>k X > since X is a unit vector 
Let Z he a unit vector colhnear with fc. Then the projection of Z on X is 
cos But the projection of k on X is proportional to the length h * of k 
Hence the projection of k on X is h k cos <t> kx We then have the theorem that 
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the projection of a test vector on any given axis is the scalar product of the test 
vector and a unit vector colhnear with the axis 

Consider a set of test vectors k and a particular test vector j We have 
seen that the projection of a test vector k on X is h k cos fkx This is the same 
as h h cos fjk, since <t>hx and <j>, k are the same angle But the correlation r ]k = 
h,h k cos <f> jk Hence the projection of the test vector k on a particular test 
vector j is 



(11) 
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Hence the correlations between a set of tests k and a ■particular test ] are propor¬ 
tional to the projections of the test vectoi s k on an axis collmear with the test vec¬ 
tor j. This theorem can be applied m the geometrical interpretation of a 
column, or row, j, of the con elation matrix Such a row shows the coriela- 
tions between a set of tests, k, and a particular test j The configuiation of 
test vectois corresponding to the conelation matrix satisfies the theorem 
that the correlations m any paiticular row or column, j, of the correlation ma¬ 
trix are proportional to the projections of the test vectoi s on an axis through the 
test vector j This is merely a restatement of the pievious theorem The 
proportionality factor is 1 /h„ so that, if the length of a particular test vec¬ 
tor j is known or assumed, then one column of the factoi matrix can be im¬ 
mediately written by (11) if we decide to locate one of the co-oidmate axes 



through one of the test vectois j. The piojcctions 1\, aie then, in fact, the 
entries of one of the columns of the factoi matux 

The same theoiem can be lestated in still another form that is impoitant 
for factorial analysis, namely, that, for any given correlation matux with an 
implied common-factor space, there exists m that space an axis through the 
origin on which the test vectois have projections that are pioportional to the 
correlations of any given column, or row, of the coi relation matux If the coi- 
relation matrix is of ordei n, there exist n such axes in its common-factor 
space 

If the co-ordinate axes are located elsewhcie in the common-factoi space, 
the projections of the test vectois on these co-oidmate axes will bo linear 
combinations of the projections on the axes through the test vectois We 
infer a fundamental theorem in factorial theory, namely, that every column 
of factor loadings m the factorial matrix is a lineai combination of columns of 
the correlational matrix We shall make use of this theoiem m designing dif¬ 
ferent methods of factoring the coi relational matrix. 

It should be clear that any interpretation of the factoi matrix must be 
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made in relation to the principles that are chosen for locating the co-ordinate 
axes in the configuration defined by the correlations In genei al, the numeri¬ 
cal entiies of the factor matrix are as arbitiary as the location of the refer¬ 
ence frame It is only to the extent that the reference frame can be in some 
sense unique that the numerical entries of the factor matrix can be given any 
scientific interpretation 

Sign reversals in the factor matrix 

When a leference frame has been inserted into the configuration and the 
projections of the test vectors computed as shown in Figure la and lb, it 
should be noted that the sense in which a eo-ordinate vector such as A m 
Figure la is taken determines the signs of the projections m column A of 
lb If the direction of the co-ordinate vector A is reversed, then all the 
signs in column A of the factor matrix are also reversed If the reference 
vector A represented some fundamental human trait, such as cheerfulness, 
then the reversal of direction of the reference vector would represent the 
direct opposite, such as grouchiness Since the scalar products of the test 
vectors are independent of the location of the reference frame, we have the 
computationally useful theorem that the signs of all the entries m a column 
of the factor matrix may be changed without altering the corresponding correla¬ 
tion matrix. Computationally, this theorem becomes evident also if we con¬ 
sider the correlation coefficient as a cross product of a pair of rows of the 
factor matrix, such as Figure lb If we reverse the signs in both a ]m and 
a km for a fixed column m, their product remains unchanged in sign so that 
the correlation coefficient r 3k remains unchanged. The psychological inter¬ 
pretation of this reversal in direction of a reference vector can be shown in 
terms of the example given The coi relation between two traits remains 
unaffected by the arbitrary decision to call one of the component reference 
traits “plus cheerfulness” or “minus grouehmess ” 

Now consider the reversal of all the signs m a row j of the factor matrix. 
This has the effect of reversing the direction of the corresponding test vector 
j. Its scalar remains the same, while its angular separation 4> }k from any 
other test vector k is changed to the supplement of 4> jk Hence the absolute 
values of the correlations of this test with the other tests remain unaltered, 
but their signs are reversed The psychological interpretation can be shown 
by an example, namely, that if one variable correlates positively with “plus 
tactfulness,” then it will correlate negatively with ‘‘minus tactfulness,” 
which might be defined as “plus tactlessness ” 

We have, therefore, another useful theorem that if all the signs ate re¬ 
versed m a row of the factor matrix, then all the signs are reversed in the corre¬ 
sponding row and m the corresponding column of the correlation matrix. To 
reverse the signs of all entries m a row j of the factor matrix does not change 
the positive sign of the self-correlation of test j, since, by the theorem, the 
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signs are reversed in the row j and also in the column j of the correlation 
matrix These two sign changes in the diagonal cell keep the sign of the self- 
correlation unaltered. This theorem can also be inferred from the factor 
matrix If all the signs in a row of that matrix are reveised, then all the 
terms in a cross product of row j with any other low k are similarly reversed 
in sign so that the correlation r ]k is reversed in sign except the self-con ela¬ 
tion of test j, which remains the sum of the squares of the entries in the 
row j We can also infer from these relations that the self-correlation of a 
test remains positive for all possible reversals of sign of tests and factors 

The restriction that r be less than n 

In the illustrative example of Figure 1 we have eight tests and two com¬ 
mon factors, so that n = 8 and r — 2 There are cei tain inequalities l elating r 
and n which must be satisfied in order that a determinate factor solution 
shall be possible To introduce these lestnctions, consider the case in which 
r>n. This case is illustrated in Figure 5, which shows a factor pattern of 
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five common factors and three tests The crosses represent non-vanishing 
factor loadings in the factoi matnx Assume that the five common factors 
are uncorrelated. The correlations between the three tests determine the 
correlation matrix, which is of older 3 The rank of that matrix cannot ex¬ 
ceed 3, but we have set up the problem to represent more than three com¬ 
mon factors If such a correlation matnx weie factored, it would give a fac¬ 
tor matrix of not more than three columns or factors The numerical entiies 
in such a factor matrix would determine the correlations exactly, but the 
three factors would probably be artifacts as far as any scientific problem is 
concerned. The relations might have some statistical interest, and they 
might be useful for constructing various empirical prediction formulae, but 
they would not reveal the five factors that were operative in the test vai i- 
ances Factor analysis is built on the postulate that the numbei of common 
factors does not exceed the number of tests in the battery. As far as a numerical 
solution is concerned, every correlation matrix can bo factoied routinely if 
we are willing to accept as many factors as there are tests. The scientific 
interpretation and validity of the factoi s is another matter 

The geometrical interpretation of the postulate that r ^ n can be seen if 
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we consider the configuration of n test vectors If these vectors are defined 
in terms of their components m more than n dimensions, the n vectors can 
also be described m an n-dimensional subspace which is spanned by the 
n vectors That is the space defined by the factor matrix, from which it 
would not be possible to identify the larger number of factors that produced 
the test correlations 

These considerations raise the question of how one can be sure that any 
particular batteiy of n variables leally does involve less than n common 
factors There is no guarantee beforehand that any proposed battery of n meas¬ 
urements will involve less than n common factors that have significant interpre¬ 
tation. To produce such a situation is as much dependent on scientific in¬ 
tuition as other scientific experiments are subject to intuition foi success 
and significance One can, of couise, factor any given correlation matrix to 
determine its rank If the rank of the coi relation matrix turns out to be com¬ 
parable with its order, one has merely made a bad guess, and one must start 
over again with a revised test battery in the same domain 

The test space and the common-factor space 

The relation between the correlation matrix and the configui ation is very 
markedly dependent on whether we insert unity or the communalities m the 
diagonal cells of the correlation matrix The principles involved here can 
be illustrated in terms of the coi relation between two tests, j and k The 
correlation matrix would be of older 2 If we write unity m the diagonal 
cells, there will be only one correlation coefficient to determine, namely, r, k 
The corresponding configuration is a pair of unit vectors, which are sepa¬ 
rated by an angle whose cosine is the given correlation coefficient If we con¬ 
sider the tests of a batteiy as represented by unit vectors to cover the total 
variance of each test, then we have a configuration of n unit vectors in a 
space of (n+r) dimensions, which represent the common factors and the 
unique factors Such a space is called the test space to distinguish it from the 
space that is defined by the common factors, which is called the common- 
factor space Since the test space represents usually more factors or param¬ 
eters than there are tests, we shall not use it except in special problems 
wheie the total test space is indicated The total test space is of n dimen¬ 
sions if we disiegard the factorial composition of the tests 

Summary of geometrical concepts 

The geometrical repiesentations descnbed m this chapter will be summa¬ 
rized here These theorems and definitions are all applicable to factor prob¬ 
lems with any determinate numbei of factors or dimensions 

The factor matrix and the configuration 

1) A reference frame must be inserted into the configuration before any 
factor matrix can be written 
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2) Since the co-ordinate axes are not determined by the experimentally 
obtained correlation matrix, every factor matnx is in the nature of a fac¬ 
torial interpretation of the given correlations 

3) Each row of the factor matnx can be interpreted as showing the rec¬ 
tangular components of a test vector 

4) Each column of the factor matrix identifies a factonal axis in the 
structure 

5) Each reference factor is represented by a unit refeience vector 

6) The giaphs on which aie plotted pairs of columns of the factoi matrix 
show the factorial structure which combines the test vectors and the refer¬ 
ence frame. 

7) If the components of an orthogonal factor matrix are all positive or 
zero, the structure has all test vectors m the first quadrant or octant. 

The correlation matrix and the configuration 

8) The correlation matrix can be interpreted as showing the scalar prod¬ 
ucts of all pairs of test vectors 

9) The length of each test vector is equal to the positive squaie root of 
its commonality 

10) The correlation matiix with known diagonal entiles dcteimines a 
unique configuiation of test vectors that can be represented as a physical 
model if the lank of the matnx is less than 4 

11) The correlation matnx does not determine any lefemu e frame 

12) If the dimensionality or lank is less than 4, a physical model can lie 
constructed to show the configuiation alone or the factorial structure which 
includes the configuration and the refeience axes 

13) No information is lost m leprcsentmg the con elation matrix as a 
configuration, or vice veisa 

14) A con elation matnx with coefficients that are all positive or zcio 
determines uniquely a configuration of test vectois in which there aie no 
obtuse angular separations 

15) The scalar product of a pair of vectors is invariant under rotation of 
axes 

16) The scalar product of a pair of unit vectors is equal to the projection 
of either vector on the other 

17) A positive coirelation coefficient lepiescnts a pair of test vectors with 
acute angular separation A negative conelation lepiescnts a pair of test 
vectors with obtuse angular separation 

18) The projection of a test vector on any given axis is the Rcalar product 
of the test vector and a unit vector collinear with the axis. 

19) The correlations between a set of tests k and a particulai test j aie 
pioportional to the projections of the set of test vectois k on an axis collinear 
with the test vector j 
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20) Given a correlation matrix with an implied common-factor space, 
there exists in that space an axis through the origin on which the test vectors 
have projections that are proportional to the correlations of any column, 
or row, of the correlation matrix 

21) Every column of factor loadings in the factor matrix is a linear com¬ 
bination of columns of the correlation matrix, 

Sign reversals m the factor minx 

22) The signs of all the entries in a column of the factor matrix may be 
reversed without altering the corresponding correlation matrix 

23) Reversing the signs in a column of the factor matrix reverses the di¬ 
rection of the corresponding reference factor and reference vector 

24) If all the signs are reversed in a row of the factor matrix, then all the 
signs are leversed in the corresponding row and m the corresponding column 
of the correlation matrix, 

25) To reverse the signs m a row of the factoi matrix reverses the direc¬ 
tion of the corresponding test vector 

26) The self-correlation of a test lemains positive for all possible reversals 
of sign of tests and factors 



CHAPTER IV 

A FACTOR PROBLEM IN TWO DIMENSIONS 

Introduction 

So far we have considered the assumptions of factor analysis, the funda¬ 
mental equations, and some of the geometiical concepts of factor theory 
In discussing the sections of Figure 1 in chapter lii we have seen the rela¬ 
tions between the factor matrix, the factorial structure, the test configuia- 
tion, and the correlation matrix We shall now consider the lllustiative 
problem of that chapter as if it were a factor problem to be solved The 
problem starts with the given correlation matux, which is oidmanly de¬ 
termined from experimental data The pioblem now is to analyze the coi- 
lelation matrix so as to discovei the factor matrix F and the structure as 
shown. The correlation matrix of the previous chapter will be used now as 
the given data, and we shall determine fiom it the factor matnx It is a 
simple matter to construct a correlation matrix from a given factor matnx, 
but it is usually more difficult to extract the factor matrix f 1 om a given coi - 
relation matrix It is the latter which constitutes factor analysis, where the 
experimental data aie m the form of con elation coefficients from which a 
factor matrix is to be found 

The solution of a faetonal problem consists usually of two steps, namely, 
(1) to factor the given correlations into a factor matnx with aibitraiy ref¬ 
erence frame and (2) to rotate the aibitraiy reference fiame into a pie- 
ferred or simplifying position The first is known as the factoring pioblem , 
and the second is known as the rotational pioblem We shall describe both 
steps for the present example There are several methods of factoring a coi- 
relation matrix, and there are several methods of solving the lotational 
problem In fact, most of the theory and computational methods of factor 
analysis are concerned with these two mam problems Here we shall start 
with perhaps the simplest of the factoring methods, namely, the diagonal 
method 

The diagonal method of factoring 

Before proceeding to the factorial solution of the con elation matnx of 
the previous chapter, we shall describe the diagonal method of factoring 
by a small five-variable matrix in Table 1 This method of factoring is not 
well suited to large problems involving fallible data, but it is a simple rou¬ 
tine method that can be used on a correlation matrix of any order and 
any rank 

The correlation matrix of Table 1 will be used to illustiate the diagonal 
method of factoring The total number of independent con elation coeffi- 
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dents is ^n(n+l) if the diagonal entries are included Since the conela- 
tion matnx of Table 1 is of order 5, theie aie fifteen correlation coefficients 


Table 1 

Correlation Matnx 



1 

2 

3 

4 

5 

1 

85 

28 

36 

82 

18 

2 

28 

80 

40 

16 

56 

3 

36 

40 

61 

54 

53 

4 

82 

16 

54 

.97 

27 

5 

18 

56 

53 

27 

58 


R 


available for determining the factorial matrix. 

The conespondmg factor matrix is shown in literal form in Table 2 and m 
numerical form in Table 3 In Table 2 all the entries above the pnncipal 


Table 3 


Factor Matrix 



r 

II 

in 

IV 

V 

1 

Oil 

0 

0 

0 

0 

2 

021 

022 

0 

0 

0 

3 

031 

032 

03 a 

0 

0 

4 

041 

042 

043 

044 

0 

5 

061 

062 

063 

064 

066 


Table 3 


Factor Matnx 



I 

II 

III 

IV 

V 

i 

922 

0 

0 

0 

0 

2 

304 

841 

0 

0 

0 

3 

390 

335 

588 

O 

0 

4 

889 

- 131 

403 

000 

0 

6 

195 

595 

433 

000 

000 


F 


F 


diagonal ai e zeros This circumstance will be explained in two ways, name¬ 
ly, geometrically m terms of the aibitrary location of a reference frame and 
algebraically in terms of the correlations expiessed as cross products of 
rows of F It should be noted that the number of coefficients to be deter¬ 
mined in Table 2 is exactly equal to the number of correlation coefficients 
in Table 1 It will be shown that each correlation coefficient m R determines 
one of the factor loadings m F, so that the method can be extended to the 
case where the rank of R is equal to its order, in which case the number of 
factors m F is equal to the number of variables in R When the rank of R 
is less than its order, it will be found that a smaller number of columns will 
be required in F In the present example the rank of R is 3, and hence the 
last two columns of Table 3 contain only zeios 

The zeio entries in Table 2 can he explained in terms of the aibitrary 
reference frame Let the first axis be colhnear with the fiist test vectoi. 
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Then this vector will have a projection on the first axis equal to the length 
of the vectoi, namely, the square root of its communality The test vector 
will have zero projections on all other possible axes that are orthogonal to 
the first axis Hence the remaining entries in the fust row of F are all zeros. 

The second axis may be placed orthogonal to the first axis and in the 
plane that is spanned by the first two test vectors Then the second test 
vector will have projections on the first two axes but not on any others 
Hence all but the first two entries in the second low of F will be zeros The 
third axis may be placed in the space spanned by the fiist tlnee test vec- 
tois and orthogonal to the first two axes The thud test vectoi will then 
have projections on the first thiee axes but not on any of the other axes. 
We are assuming that the successive tests that aie used for the location of 
the reference frame are so chosen by inspection that they aie lincaily mde- 
dependent The manner of doing this will be shown in the numencal ex¬ 
ample The fourth axis is placed orthogonal to the first tlnee axes and in 
the four-dimensional subspace that is spanned by the fust four test vectois 
Hence the fourth test vectoi will have piojections on the first four oithogo- 
nal axes but not on any of the other axes This process is continued until a 
sufficient numbei of dimensions has been defined to account for the scalar 
products of the correlation matnx This location of the lefeience frame 
produces a set of zeio entries in the uppei half of the faotoi matnx, as 
shown in Table 2 Tlie zeio entries of this table can be inseitcd at the stait 
of the factoring 

The triangulai airangement of zeios in Table 2 can also be explained 
algebraically m terms of the cioss products of F that must lcpioduce the 
correlations in R The sum of the squares of the fiist low of Table 2 is 
equal to the communality of test 1. Since there is only one entiy in the 
fiist row of F, wc have 


an = V.85 = 922 , 
and this is lecorded in Table 3 

The cross product of the first two lows of F must lepioduce the coirela¬ 
tion r 2l = 28 Hence we have 


anflai — 28 , 

922a2i = 28 , 
a 21 = .304 . 

the correlation between the first test and any other test j becomes 

(1) 


i — r,i, 
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OT 

( 2 ) 


0,1 


hi 

On 


922 rjl 


The entries in the first column of Table 3 are therefore proportional to the 
fiist column of correlation coefficients in R, as shown in equation (2) 

When the first column of the factor matrix has been recorded, we turn to 
the communality of the second test, which is the sum of the squares of the 
entries in the second row of F Hence 


ah 4- a« = 80 , 


in which an is the only unknown It can therefore be determined, and it is 
recorded as 841 in Table 3 

The corielations of the second test with any other test j can be written 
in the form 


in which a,i is the only unknown The second column of Table 3 is thus 
numerically determined 

The communality of the third test is, similarly, 

3 

' 7*33 ; 

m=l 

in which a 3 3 is the only unknown. It is recorded as 588 in Table 3 

The correlation of test 3 with any other test j can be written 

3 

U3 m (Zj m — , 

m=l 

in which a ,3 is the only unknown Hence the third column of Table 3 be¬ 
comes numerically determined 

In this numencal example, the rank of R is 3, and hence, if this process 
is continued, the remaining factor loadings in Table 3 will be found to van¬ 
ish Three columns of F represent three arbitrary reference factors, which 
account for the correlations in R 

In the diagonal method of factoring, the selection of each successive test 
should be made on the basis of the residual variances of the tests For ex¬ 
ample, when two columns of Table 3 have been numerically determined, 
one might compute the sum of the squares of each row of F and compare 
these sums with the communalities as shown in R That test might be taken 
for determining the third axis, which shows the largest residual variance, 
but any test can be used as a pivot for the next factor if it has an appre¬ 
ciable diagonal entry 
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The practical limitation of the diagonal method of factoring foi fallible 
data is that the method pivots on one test at a time The choice of a diagonal 
entiy which is estimated with practical data deteimines the faetoi loadings 
in a manner which is veiy sensitive to errois in the estimations foi the 
diagonal cells In general, computing methods are prefen ed in which all 
the data contribute to the deteimination of each parametei by summations 
However, the diagonal method is theoretically coirect, and it can be used 
as a loutme computational method of factonng where the given coriela- 
tional entries can be legalded as exact. 

A two-dimensional example 

The correlation matrix m Figure 1-ni has been reproduced lieie as 
Table 4, and it will be factoied by the diagonal method We choose arbi- 


Table 4 

Con elation Matrix 



1 

2 

3 

4 

5 

0 

7 

8 

1 

58 

63 

49 

.51 

35 

18 

45 

24 

2 

63 

81 

36 

54 

45 

00 

27 

00 

3 

46 

36 

52 

42 

.20 

36 

60 

48 

4 

51 

54 

42 

45 

30 

18 

42 

24 

5 

35 

45 

20 

30 

25 

00 

15 

.00 

6 

18 

00 

36 

18 

00 

36 

48 

48 

7 

45 

27 

60 

42 

15 

48 

73 

64 

8 

24 

00 

48 

24 

00 

48 

64 

04 

2 

3 40 

3 06 

3 40 

3 06 

1 70 

2 04 

3 74 

2 72 


tianly the first test to detei mine the fust refeience axis The commonality 
of test 1 is 58, and hence the desired multiplier is l/V 58=1 313064 
Applying this stretching factor on the first column of conclations m Table 4, 
we obtain the first column of factor loadings which are entered in Table 8 
The computational checks have been included in this example First, we 
sum the columns of Table 4, as shown in row 2 The sum for the first col¬ 
umn, 3 40, is treated as an additional entry in this column Applying the 
stretching factor to it, as for the correlation coefficients in the first column, 
we have 4 464, which is recorded in Table 8, low Ch The actual sum of 
that column of Table 8 is 4 465 Since the predicted check sum (Ch) agrees 
with the actual sum (2) in Table 8, we assume that the computations and 
the copying into the table are correct Wc shall not specifically mention 
each check sum, since the piinciplc is essentially the same foi the rest of 
the computations. Almost every novice in computing minimizes the im¬ 
portance of foimal computational checks Aftei he has wasted some days 
hunting for his errors, he learns that it saves both time and annoyance to 
proceed with formal explicit checks at every step in computing In the 
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publication of computational results it is not customary to distract the 
reader with the computational checks. 

When the first column of the factoi matux m Table 8 has been numencal- 
ly determined, the cross products of this column are listed as shown m 
Table 5 The summation in Table 8 is heie again treated as an additional 
entry, which gives row Ch in Table 6, This table shows the contributions 
made by the first reference factor to the given conelations 

Table 6 

First-Factor Products a^au 



l 

2 

3 

4 

5 

6 

7 

s 

1 

581 

630 

460 

511 

351 

180 

450 

240 

2 

630 

684 

500 

554 

380 

195 

.489 

261 

3 

460 

500 

365 

405 

278 

143 

357 

190 

4 

511 

554 

405 

449 

308 

158 

396 


5 

351 

380 

278 

308 

212 

109 

272 

145 

6 

180 

195 

.143 

158 

109 

056 

139 

074 

7 

450 

489 

357 

396 

272 

139 

349 

186 

8 

240 

261 

190 

211 

145 

074 

186 

.099 

Ch 

3 402 

3 693 

2 697 

2 992 

2 054 

1 054 

2 639 

1 406 

2 

3 403 

3 693 

2 698 

2 992 

2 055 

1 054 

2 638 

1 406 


Table 6 

Firsl-Faclor Residuals n n. — r,* — a, t ati 



1 

2 

3 

4 

5 

6 

7 

8 

1 

- 001 

000 

000 

- 001 

- 001 


000 


2 


126 

- 140 

- 014 

■wi 

- 195 

- 219 

- 261 

3 


- 140 

155 

015 

- 078 

217 

243 

290 

4 

- 001 

- 014 

015 

O01 


022 

024 

029 

5 


070 

- 078 

- oos 


- 109 

- 122 

- 145 

6 


- 195 

217 

022 

- 109 

304 

341 

406 

7 


- 219 

243 

024 

- 122 

341 

381 

454 

8 

000 

- 261 

290 

029 

- 145 

406 

454 

541 

Ch 

- 003 

- 633 

702 

068 

- 355 

986 

1 102 

1 314 

2 

- 003 

- 633 

702 

068 

- 355 

986 

1 102 

1 314 


The cross products shown in Table 5 aie subtracted from the correla¬ 
tions in Table 4 These are denoted 

which are shown m Table 6 The subscript 2 in rt lefeis to the fact that 
these are residual conelations fiom which the second factoi loadings aie 
to be deteimmed * Table 6 can now be treated just like Table 4, in ordei 
to determine the next column of factor loadings 

* In some of our previous work wo have used the notation 7*2 to represent second- 
factor residuals instead of first-factor residuals, as in this problem 
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Column 8 m Table 6 shows the highest diagonal entry, and we therefore 
choose it as the second pivot test The residual c omm unality for test 8 is 
541, so that the multiplying factor is now 1/V 541 = 1 359569. Apply¬ 
ing this stretching factor to column 8 of Table 6, we get the second column 
of the factor matrix m Table 8 


Table 7 

Second-Faclor Products a yl an 



1 

2 

3 

1 

5 

6 

7 

8 

1 


000 

000 

000 

000 

000 

000 

000 

2 

000 

126 

- 140 

- 014 

070 

- 196 

- 219 

- 261 

3 


- 140 

155 

015 

- 078 

217 

243 

290 

4 

000 

- 014 

015 

002 

- 008 

022 

024 

029 

5 


070 

- 078 

- 008 

039 

- 109 

- 122 

- 145 

6 

000 

- 196 

217 

022 

- 109 

305 

341 

.406 

7 

000 

- 219 

243 

024 

- 122 

341 

381 

454 

S 

000 

- 261 

290 

029 

- 145 

406 

454 

542 

Ch 

000 

- 634 

704 

070 

- 352 

980 

1 102 

1 314 

s 

000 

- 634 

702 

070 

- 353 

980 

1 102 

1 315 


Table 8 


Factor Matrix F 


Table 9 



I 

II 

1 

762 

000 

2 

827 

- 355 

3 

604 

394 

4 

670 

039 

5 

460 

- 197 

6 

.236 

552 

7 

591 

617 

8 

315 

736 

Ch 

4 464 

1 786 

S 

4 465 

1 786 


d 68 641 


v'rf 761577 735527 

1/VSl 313065 1 359569 


Given Factor 


Transformahon Rotated, Factor 


Matrix Mati'ix Matrix 



I 

II 


A 

n 


A 

B 

1 

762 

000 

I 

Xi fl 

Xj b 

1 

70 

30 

2 

827 

- 355 

II 

X 2 a 


2 

90 

(X) 

3 

604 

394 




3 

40 

60 

4 

070 

039 




4 

00 

30 

5 

460 

- 197 

=— 

- — 


5 

50 

00 

6 

236 

552 




6 

00 

00 

7 

591 

617 


A 

B 

7 

30 

80 

8 

315 

736 




8 

00 

80 




1 

917 

400 







II 


917 












l'\ 


_ 




v\ 



A 


The cross products of the second column of the factoi matux give 
(ojsOm), which are shown in Table 7 These are the contributions of the 
second reference factor to the conolations in Table ^ Subti acting the en¬ 
tries of Table 7 from the first-factor lesiduals r 2lk in Table 6, we find that 
they vanish, showing that the two factors m Table 8 are sufficient to ac¬ 
count for the given coi relations 
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Rotation in two dimensions 

The factor matrix of Table 8 has been plotted m Figure 1, wheie we see 
the two-dimensional configuration and the arbitrary reference axes I and II 
In choosing a leference frame for the intei pretation of the variables m terms 
of factors, we must decide on some criterion by which to make the choice 
The fit st axis might be passed through the centroid of the eight points in 
Figure 1, or the fust axis might be so placed as to maximize the sum of the 
squaies of the projections of the eight test vectors We might place the 
axes in such positions that the negative projections will be eliminated or 
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Figure 1 


minimized as far as possible. Another criterion which can sometimes be 
used is to place the axes so as to minimize what we have called the “com¬ 
plexities” of the tests When this criterion is applicable, the axes are so 
placed that a large number of zero entries appear in the factor matrix. The 
rotational problem is one of the most important m factor analysis 

In the present example we see in Figure 1 that the angular span of the 
configuration is a right angle, so that a frame can be chosen which eliminates 
all negative factor loadings and which i educes the complexities of four 
tests to one instead of two The two axes chosen by these criteria are de¬ 
noted A and B in the figure. 

The numeucal problem now is to write a factor matrix which will repre¬ 
sent the two orthogonal factors A and B instead of the arbitrary orthogonal 
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factors I and II, which appeared in the factoring of the coi relation matrix 
In Table 9 we have the factor matrix F i for the axes I and II (from Table 8), 
and we want to find an orthogonal transformation matrix A so that F iA = 
Fi, where F 2 will be a factor matrix for the axes A and B * The notation 
Fi, F 2 , F 3 , . , will henceforth be used to repiesent successive rotations 

of the reference fiame 

Since the present configuration of test vectors is contained in two dimen¬ 
sions, the orthogonal transformation A will be of order 2X2 It will be 
useful to make a geometrical interpretation of the cell entries in the matrix 
A of the transformation The cell entries in the first column A of this matrix 
are denoted Xi a and X 2a These are the direction cosines of the unit vector A 
referred to the frame I—II Another way of saying the same idea is that the 
unit vector A in Figure 1 can be expressed as a vector sum of its projections 
on the axes I and II. We have, then, for the vector A, 

(4) A — X la I -f- \ 2a II. 

The first column A of the orthogonal matrix A in Table 9 states that the 
vector A is X ia of the unit vector I plus X 2a of the unit vector II In Figure 1 
these coefficients of equation (4) can be determined graphically Since A is 
a unit vector, we shall expect that 

(5) XL + XL = 1 

By inspection of Figure 1, and normalizing, we have 

(6) A = .917 I - .400 II, 

and these coefficients are entered in the numerical form of the transforma¬ 
tion matrix A in Table 9 

A similar interpretation applies to the second column B of the transfor¬ 
mation matrix From the figure we obtain close estimates of the projec¬ 
tions of B on I and II, namely, Xu, and X 2 &, which, when normalized, give 
the unit vector B as the vector sum, 

(7) B = .400 I + .917 II . 

In Table 9 it should be noted that the rows of the transformation matnx 
refer to the axes I and II, while the columns refer to the axes A and B If 
we perform the matrix multiplication 

(8) FiA = F 2 , 

* For theorems related to rotation of axes see "Mathematical Introduction” and 
theorems in chap tti 
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the pi o duct is of order nX 2, and the columns are denoted A and B This 
has been done m Table 9, where F 2 is the desired factor matrix for the fac- 
tonal axes A and B. This matrix is identical with the factor matrix with 
which we started in chapter m 

In wilting this factorial solution, we might have labeled the two axes A 
and B m leveise oidei m the figure, or we might have reversed the order of 
the two columns of the transformation matrix A. The effect would have 
been meiely that the columns of l<\ would appear in a different Older It 
should be evident that the older m which the columns of are written 
would have no effect on the interpretation of the factors in teims of the tests 

The extension of the reasoning illustrated here to three or moie dimen¬ 
sions involves essentially no new ideas, although the higher-dimensional 
pioblems are more complex In problems involving more common factons 
it is moie difficult, for example, to identify the nature of the configuration, 
which is obvious jn a single diagiam in two dimensions 



CHAPTER V 

THE GROUPING METHOD OF FACTORING 

The diagonal method of factoi mg a con elation matnx has been clcsei lbod 
in chapter iv While that method is simple in principle, it has a senous dis¬ 
advantage when dealing with fallible data, namely, that it pivots succes¬ 
sively on single tests and on single coi 1 elation coefficients If any of these 
individual coefficients is m oiror, the calculations that aie based on them 
are similarly in eiror, so that the residuals become laigei than when some 
summational piocedure is used For this leason the centroid method was 
developed The giouping method to be described heie is a vaunnt of the 
centroid method 
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Figure 1 

Consider the correlation matnx r ]k m Figuie 1 Instead of summing 
over all the variables, which is the stai ling-pomt foi the centioid method, 
we shall sum here over a group of tests The group should contain at least 
thiee or four tests The successive groups to be used should be so selected 
that the mterconelations of the tests within one gioup are laige, lelativc 
to the magnitudes of the coefficients m the table Detailed loutme for 
selecting the gioups will be shown in the numencal example Let thcie be s 
tests m the selected gioup Let the lows of the selected gioup be denoted 
by the subsenpt u, and let the coi responding columns be denoted by the 
subsciipt v Let the lank of the correlation matnx be r 

Let us augment the correlation matnx by a row, as shown m Figure 1, 
of which each element, t k} is the sum of the coefficients m the w-rows foi 
column k These elements arc 

a) a = 
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where r uk represents the correlation coefficients m the selected rows The 
augmented matrix is now of order (n+1) by n It should be noted that the 
rank of the matrix is unaltered became the new 1 ow is a linear combination 
of the rows of the given matrix r,k 

Next we augment the matrix by a column, as shown in the figuie Each 
element t, of the new column is the sum of the coefficients in the v columns 
of the row j These elements are 

a 

( 2 ) ]>>„ = t Jt 

» = 1 

wheie r JV represents the correlation coefficients in the selected columns 
The augmented matrix is now a square matrix of older (n+1) Its rank is 
still unaltered because the new column is a linear combination of the col¬ 
umns of the previous matrix 

The diagonal entiy at the mteisection of the new column and the new 
row is denoted T Let t u and t„ denote the sums t, and t k in the rows and 
columns of the selected group Then T can be wntten 


(3) 

because the transpose of the row t k is the column t, 
Let a multiplying factor be defined by 


(4) 


m m 


1 

Vt 


I hen a new matnx may be written as in Figure 2 , which contains the squaie 
matrix r lk with a new row a k and a new column a, and then mteisection 
The entries, a, in the new row are determined by the relation 

® «r = mt k , 

a,nd the diagonal entry then becomes Vt The entries in the column a, aie 
determined by the relation 

a, — mtj, 

after which the diagonal entry becomes 1. The rank of the matrix is un¬ 
affected by the application of the sketching factor m The column a, is 
the transpose of the row a k , since a, = a k when .7 = 7 c 

The new row and column can now be regarded as showing the projec- 

t ‘° ns ° f ” tests ? f the matrlx Ur on a unit reference vector I, as shown in 
Figure 2 The entries a, can therefore be taken as the first column of the 
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factor matrix A table of first-factor residuals is then prepaied, and a new 
group of relatively well-correlated tests selected for a lepetition of the same 
process This proceduie is continued until the residuals disappear or be¬ 
come small enough to be ignoied 

As a numencal example for factoring we shall take the con elation matrix 
of Figure 1 m chapter m That matrix is reproduced here in Table £ We 
select a test that has the largest number of relatively high coefficients. 
Such is No, 7, which has three coefficients of 60 or highei We arrange the 
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Table 1 
First Group 



7 

s 

3 

7 

73 



8 

64 

64 


3 

60 

48 

.52 

6 ‘ 

.48 

48 


1 

46 


46 

4 

42 


42 


coefficients foi No. 7 in descending order by absolute value and tabulate the 
mtercorrelations of a group of tests as shown in Table 1. In this table aie 
shown only the coefficients above 40 What is to be regarded as a high 
coefficient depends on the order of magnitude of the coefficients m the whole 
table The grouping method does not give a unique solution, and there is 
no absolute rule for determining the size of the group of tests to be used for 
computing each successive column of the factor matnx In Table 1 we se¬ 
lected three tests to constitute the first group, namely, 7, 8, and 3. We 
might have included test 6 

The three tests selected to constitute the first group are marked in col¬ 
umns and rows of Table 2 The summations 4 are shown for each column, 
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as well as the sum T Applying the multiplying factor m, we have the bot¬ 
tom row of the table, which shows the fiist-factor loadings These are copied 
as the first column of the factoi matrix in Table 6 

In Table 4 aie shown the fiist-factor residuals These are (r }k — a 3 ia kl ) 
These entries aie obtained by subtracting elements of Table 3 fiom the 
conespondmg elements of Table 2 

Table 2 


Correlation Matrix 





X 




X 

X 


1 

2 

3 

4 

5 

6 

7 

8 

1 

58 

63 

46 

51 

35 

18 

45 

24 

2 

63 

81 

36 

54 

45 

00 

27 

00 

X 3 

46 

36 

62 

42 

20 

36 

60 

48 

4 

51 

54 

42 

45 

30 

18 

42 

24 

5 

35 

45 

20 

30 

25 

00 

35 

00 

e 

18 

00 

36 

18 

00 

36 

48 

48 

X 7 

45 

27 

60 

42 

.15 

48 

73 

64 

X 8. 

24 

00 

.48 

24 

00 

48 

64 

64 

( 

1 15 

63 

1 60 

1 08 

35 

1 32 

1 97 

1 76 

I 

498 

273 

693 

468 

152 

572 

853 

762 


Table 3 

Fvst-Factor Products 



1 

2 

3 

4 

6 

6 

7 

8 

1 

248 

136 

345 

233 

076 

285 

425 

379 

2 

130 

075 

189 

128 

041 

156 

233 

208 

3 

345 

189 

480 

324 

105 

396 

591 

528 

4 

233 

128 

324 

219 

071 

268 

399 

357 

5 

076 

041 

105 

071 

023 

087 

130 

116 

6 

285 

156 

396 

268 

087 

327 

488 

436 

7 

425 

233 

591 

399 

130 

488 

728 

650 

8 

379 

208 

528 

357 

116 

436 

650 

581 


In order to compute second-factor loadings, a column is chosen which 
has a relatively laige number of the highei coefficients Test 2 is so selected, 
and its coefficients aie arranged m descending oidei of absolute values m 
the first column of Table 5 The lowei coefficients are ignored The mter- 
corrclations are recorded as shown Test 8 has a negative correlation with 
test 2, so that test 8 is recoided heie with negative sign ( — 8) The con ela¬ 
tion between tests (+2) and (—8) is -|- 208 A similar interpretation applies 
t,o the othei coefficients m this table A gi oup of tests is to be selected in 
which the signs of the mteicorrelations within the group aie all positive 
Accoiding to this criterion, all the tests in Table 5 could be included m the 
gioup, with tests G and 8 taken as negative The coefficients of seveial 
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tests are rather low, so we retain only four tests for the second group, name¬ 
ly 2, 1, 4, 5 These are marked m rows and columns of Table 4 

The iow ti is determined by summation of four coefficients in each col¬ 
umn, and the value of T is computed as shown Applying the multiplying 
factoi in, we obtain the second-factor loadings in the bottom row, II These 
are copied as the second column of the factoi matrix of Table 6. 

Table U 

First-Factor Residuals 



X 

X 


X 

X 





1 

2 

3 

4 

5 

0 

7 

8 

X 1 

332 

494 

115 

277 

274 - 

- 105 

025 - 

- 139 

X 2 

494 

735 

171 

412 

409 - 

- 156 

037 - 

- 208 

3 

115 

171 

040 

090 

095 - 

- 036 

009 - 

- 048 

X 4 

277 

412 

09G 

231 

229 - 

- 088 

021 - 

- 117 

X 5 

274 

409 

095 

229 

227 - 

- 087 

020 - 

- 116 

6 

- 105 

- 156 

- 036 

- 088 

- 087 

033 - 

- 008 

044 

7 

025 

037 

009 

021 

020 - 

- 008 

002 - 

- 010 

8 

- 139 

- 208 

- 048 

- 117 

- 116 

044 - 

- 010 

059 

i 

1 377 

2.050 

477 

1 149 

1 139 - 

- 436 

103 - 

- 580 1 

11 

576 

858 

200 

481 

476 - 

- 182 

043 - 

- 213 


Table 5 
Second Group 



+ 2 

+ 1 

+ i 

+ 5 

-8 

+ 3 

+2 

735 






+ 1 

494 

332 





+4 

412 

277 

231 




+5 

409 

274 

229 

227 



-8 

208 

139 

117 

116 

059 


+3 

171 

115 

096 

095 

048 

040 

-6 

156 

105 

088 

087 

044 

036 


Table 0 
Factor Matrix 



I 

II 

1 

498 

570 

2 

273 

858 

3 

693 

200 

4 

408 

481 

5 

152 

476 

6 

572 

- 182 

7 

853 

043 

8 

762 

- 243 
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When the second-factor products (a^ati) are subtracted from the first- 
factor residuals in Table 4, it is found that the second-factor residuals all 
vanish The reason for this is that the given matrix of Table 2 is of rank 2 
The grouping method is exact in the sense that if the given correlation 
matrix is of rank r, then the extraction of r factors by this method will give 
residuals that all vanish identically. This is also true of the centroid meth¬ 
od to be described later 

In Figure S we have plotted the factor loadings of Table 6. The configura¬ 
tion here is the same as that of Figure la in chapter in The apparent dif- 


I 



ference is due to the interchange of abscissae and ordinates This is asso¬ 
ciated with the fact that in multiple-factor analysis there is no distinction 
between independent and dependent variables. The arrangement of the 
columns in the factor matrix is of no significance, and, in plotting two- 
dimensional sections of a configuration, it is of no consequence whether 
any particular factor is plotted as abscissa or as ordinate An orthogonal 
transformation of order 2 can be written, as previously described, by which 
lable 6 is carried back to the factor matrix of Figure lb in chapter m. 

The grouping method has been used on large factor studies It is com¬ 
parable with the centroid method Each has certain advantages that will 
e discussed in a later chapter. The check rows and columns were omitted 
rom the present chapter m order that the principles of the method might 
be described without the distractions of computational detail 



CHAPTER VI 

FACTORS AS EXPLANATORY CONCEPTS 


Factor analysis was developed as a scientific method of investigating re¬ 
lated measurements that are not well understood In order to illustrate the 
method we shall apply it to a problem m which the relations between the 
measurements are simple and well known. Instead of dealing with human 
beings and their test scores or other measurements, we shall consider a set 
of simple physical objects In Table 1 we have listed twenty-seven cylin- 

Table 1 


Twenty-seven Cylinders 


Cylin¬ 

der 

d 

2 

a 

Cylin¬ 

der 

d 

l 

8 

Cylin¬ 

der 

d 

! 

8 

i 

1 

2 

i 

10 

i 

2 

2 

19 

i 

2 

3 

2 

2 

2 

i 

n 

2 

2 

2 

20 

2 

2 

3 

3 

3 

2 

i 

12 

3 

2 

2 

21 

3 

2 

3 

4 

1 

3 

i 

13 

1 

3 

2 

22 

1 

3 

3 

5 

2 

3 

i 

14 

2 

3 

2 

23 

2 

3 

3 

6 

3 

3 

i 

15 

3 

3 

2 

24 

3 

3 

3 

7 

1 

4 

i 

10 

1 

4 

2 

25 

1 

4 

3 

8 

2 

4 

i 

17 

2 

4 

2 

26 

2 

4 

3 

9 

3 

4 

i 

18 

3 

4 

2 

27 

3 

4 

3 


ders The shape of each cylinder is completely determined by two parame¬ 
ters, namely, the diameter d and the length l For each cylinder is also 
listed the specific gravity s 

With these given measures we can compute for each cylinder the follow¬ 
ing seven measures 


1) diameter d, 

2) length l 

3) base aiea a , 

4) side area c , 

5 volume v , 

6) diagonal t, 

7) weight w . 


ir d 1 2 3 4 * 6 7 


c = irdl ; 

7T dH 

v = IT ■ 


t = Vd* + P , 

W = SV 


The computed values of these measures were correlated The result is 
shown in the correlation matrix of Table 2 for the seven measuies on the 
population of twenty-seven cylinders The diagonal entiies were computed 
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as the best-fitting values for rank 2 by methods to be described m later 
chapters The correlation matrix of Table 2 will now be regarded as the 
starting-point for a factor analysis m which we shall try to extract as much 
infoimation as possible about these seven measures solely from the vnter- 
correlations 

We shall considei this pioblem as if the actual cylinders were available, 
each one identified by its numbei and by its seven scores But we shall as¬ 
sume that wc know nothing about how each of the measurements was taken 
and that we know nothing about how the measuiements happen to be cor¬ 
related For the purposes of this lllustiative example, this situation corre¬ 
sponds to that in which each member of an experimental population is 
available and is identified as to his several test scores or other measurements 
but in which we do not understand what the measurements mean or why 
the measurements happen to be coi related 

Table H 

Coi relation Mali ix 



1 

2 

3 

4 

5 

6 

7 

1 

978 

000 

990 

812 

895 

656 

746 

2 

ono 

1 000 

000 

541 

348 

823 

290 

3 

990 

000 

987 

803 

905 

558 

754 

4 

812 

541 

803 

968 

969 

874 

807 

5 

895 

348 

905 

969 

962 

767 

834 

« 

550 

823 

558 

874 

767 

968 

639 

7 

746 

290 

754 

807 

834 

639 

672 


Before starting the faetoiing, we inspect the intercom elations that have 
been given to us foi analysis Perhaps the first thing to be noticed is that 
all the intercorielations in this problem are positive oi zeio This means 
that the configuration of test vectois does not spread over moie than a 
right angle and that all the test vectors are contained within a cone of 45° 
geneiatmg angle. This is also the situation actually found expei mentally 
with tests of the cognitive functions 

Inspection of the diagonal cells shows lemarkably high communalities 
except foi vauable No 7, whose communality is 672 In most expenmental 
studies one does not find communalities of .06 and higher unless the domain 
has been explored so completely that most of the vauance of each test is 
fairly well known Variable 7, whose communality is 672, has only two- 
thnds of its variance m common with the other tests of this batteiy Hence 
one-thnd of its variance is unique In this example it is easily seen why 
this should happen, but, in factorial studies generally, the nature of unique 
vauance is usually unknown Vaiiable 7 is the weight of the cylinder, which 
is detei mined by the volume and the specific gravity The volume is a func¬ 
tion of diameter d and length l , and hence the vauance in this characteristic 
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is shared with the other measurements, which are also functions of d and l 
But the specific gravity s enters into only one variable of the batteiy, 
namely, the weight w, and hence the individual diffci cnees among the cylin¬ 
ders as regards specific gravity produce unique variance in w That pait 
of the variance of w which is due to volume is common-factor vanance, 
while that pait of the vanance of w which is due to specific giavity is unique 
variance The common-factoi variance of w is its communality, while the 
unique variance is the complement of the communality 

The con elation matux of Table 2 was factored by the gioupmg method 
that was described in chapter v Tests 1, 3, 4, 5, were used for the first gioup, 
and tests —1, +2, —3, +4, +5, +6, were used for the second factoi. The 
resulting factor matrix is shown in Table S, a This factoi matnx was plotted 
m Figuie, 1, with the orthogonal axes I and II as shown 


Table S 

Factor Matrices 



a 



b 



C 



I 

II 


Ai 

Az 


Jil 

Hi 

1 

960 

- 234 

1 

989 

000 

1 

821 

- 551 

2 

232 

972 

2 

000 

1 000 

2 

554 

835 

3 

963 

- 238 

3 

992 

- 004 

3 

822 

- 556 

4 

928 

341 

4 

821 

551 

4 

089 

000 

S 

975 

132 

5 

916 

359 

5 

961 

- 213 

0 

720 

643 

6 

547 

796 

6 

898 

355 

7 

821 

119 

7 

770 

310 

7 

812 

- 172 


With the factoung completed, we have befoie us the factoi matnx of 
Tables, a, and the configuration of Figure 1 We also have the twenty-seven 
cylinders, each identified with its seven measures or scoies, but we have, 
supposedly, no knowledge of what the measuioments mean In this simple 
geometrical example we hope to show the fundamental nature of factonal 
analysis as a scientific method If the principles can be seen m a simple case 
in which the underlying lelations aic definite and well known, pci haps the 
application of the method to mote complex pioblems can be undo taken 
with more confidence and with a bettei understanding of its possibilities 
and limitations 

It has been shown that the location of the lcfeience fiame I- II that is 
obtained in factoring is essentially arbitraiy Its location in the configura¬ 
tion depends on the method of factoring. In the present case the axes / 
and II were detei mined when the successive gioups of tests were chosen 
The method of choosing the groups is not unique Hence we should not be 
misled in assuming some simple physical interpretation of factois I and II 
They may have physical meaning if each gioup constitutes a clustei with 
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some trait in common, In the present ease there is no definite cluster about 
axis I, and there is no cluster at all near axis II 

The seven test vectors in Figure 1 spread in a fan fiom tests 1 and 3 to 
test 2, with the other test vectors between them. It can be seen by inspec¬ 
tion that the fan covers a right angle This fact is associated with the fact 
that test 2 correlates zero with 1 and 3 as shown m Table 2 Tests 1 and 3 
are practically collinear, and this is associated with the fact that their corre¬ 
lation is practically unity 

One alternative is to locate the reference frame at A i and A 2 because this 
orthogonal pair of reference vectors incloses the whole configuration If 
we put the reference frame m this position, the factor loadings will be all 



Figure 1 
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positive or zero, as shown m the factor matrix of Table 8, b. A factor matrix 
of this type and of rank 2 is associated with the fact that all the intercom e- 
lations are positive or zero The extension of this relation to 3 and higher 
dimensions will be discussed later An interpretation can often be found 
for factors, such as A 1 and Ai, which inclose a whole configui ation. 

Another alternative location of the reference frame is to put the fiist 
axis in the middle of the configuration. Such an axis can be located in the 
middle of the span between A\ and A 2 , or it can be so located that it will 
pass through the centroid of all the test-vector termini, or it can be located 
so as to maximize the sum of the squares of the projections of the test vec¬ 
tors on the axis The two latter methods would give nearly the same factor 
axis m Figure 1. As an example of the location of a first axis in the middle 
of the configuration, we have drawn the factor axis B y through test 4 The 
resulting factor loadings are shown in the first column of Table 3, c It will 
be seen that the factor loadings are all positive and of appi eciable magni¬ 
tude A common factor which has appreciable loadings on all the tests in a 
battery is called a general factor for that battery Stiietly speaking, a general 
factor might be one that has non-vanishing loadings in all tests, even 
though some are positive and some negative But, as the term is ordinarily 
used, a general factor is one that has appreciable positive loadings in all 
tests of a battery In the present problem the location of a general-factor 
axis is not unique because it can be moved over a considerable range and 
still give appreciable positive loadings for all tests If a general-factor axis 
Hi is placed somewhere in the middle of the configuration, the second or¬ 
thogonal factor axis B v . can be placed as shown or in the opposite direction 
One of these directions is chosen as positive (-f B 2 ) and the othei as nega¬ 
tive (-B 2 ) The same could have been done for the first axis—if there had 
been any test vectois with negative projections The factor loadings on B 2 
are shown in Table 3, c It will be seen that some tests have positive load¬ 
ings and that some tests have negative loadings on the faotoi B 2 A factor 
which has both positive and negative saturations is called a bipolar factor. 

It should also be noted that whenever a factor axis is placed in the middle 
of a configuration, then each of the subsequent factors m the analysis will be 
bipolar factors This theorem is clearly seen in the configuration of Figure 1, 
and it is true for a test configuiation in any number of dimensions or fac 7 
tois. Since this relation follows as a geometrical necessity from the location 
of the first axis in the middle of the configuration of test vectors, we should 
be cautious in attempting to impose a physical interpretation on the bi¬ 
polarity of factors This type of bipolarity may have physical significance, 
but it may also be merely a geometric consequence of the location of the 
first axis in the middle of the test configuration 

We have considered several of the principles by which a frame of refer¬ 
ence factors can be located in the configuration We shall now proceed to 
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analyze two of these reference finines to illustrate the manner in which one 
attempts to find an interpictation for each reference factor 

The frame AiA 2 is represented by two unit vectois, Ai and Ai Each of 
these unit vectois may be thought of as a test vector for a perfect test of a 
factor Such a test would have unit communality The factor Ai is of a 
natuie such that it deteimmes the mterconelations of measures 1 and 8 
completely, since these two measures have zeio loadings on the factor Ai 
The factor Ai is entirely absent m the variance of test 2, since the factoi 
loading is zero The factor A i is involved m the variance of all the measures 
to an appreciable extent except No 2, but it is lopiesented in. almost pure 
foim in tests 1 and 3 

If we know something about the measuies that are involved, we turn to 
them to see if some hypothesis can be formulated foi the factor that would 
seem plausible for all the tests that are high in the factoi and which would 
also be absent in tests that are low m the factor In psychological woilc one 
can sometimes make a plausible hypothesis by examining the tests, but 
more often it is necessary to use mtiospection for each test to asceitam the 
nature of the common factor This is the method that has been used for 
identifying most of the primal y abilities 

In the present example it is assumed that we have no knowledge about 
the nature of the measurements This is even more of a handicap than we 
oidinaiily have in factorial studies, because experimental measurements aie 
usually based on some knowledge of their implications, even if their under¬ 
lying order is not understood If we anange the nine different cylindrical 
forms in order, from the lowest to the highest in accordance with factor A i, 
we have the anangement shown in Ftgui e 2 This airangement reveals that 
the factor is associated with the thickness of the cylinders, and we might 
call it the "thickness” factor The diameter (I) and the moss-sectional 
area (3) are both deteinuned by this factoi, as is shown by the fact that 
these two measurements have no piojection on the only other common fac¬ 
toi m this battery. The fact that the communahties aie near unity reveals 
that there is liaidly any residual oi unique vanance left m these two meas- 
uiements aftei the factor A i has been deteimmcd 

We pioceed next in the same manner with the factoi A 2 by airanging the 
nine different cylindrical forms m oidei, as shown in the figuie These 
cylinders are evidently arranged m the oi der of their height, and we might 
therefoie call this the “height” factor This factor is involved in all the 
measuies except 1 and 3 It should be noted that, when we name these fac¬ 
tors either by examining the nature of the measurements m some way or by 
examining the individual members of the experimental population, we have 
only a hypothesis concerning the natuie of the factoi Sometimes two or 
more hypotheses appear to be equally plausible for the interpretation of a 
factor Then a new factoi lal experiment should be set up, with additional 
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measurements so selected that they represent the two hypotheses with the 
least possible overlapping The resulting factor) al expeiiment might sustain 
one of the hypotheses, or it might show that neither of the hypotheses is 
correct In the present case we have identified one factor that is associated 
with thickness. It has two scores or measurements completely determined 
by this factor, namely, 1 and S (diameter and base area) Anothei factor is 
associated with the height of the cylmdei We have found that all the 
measurements involve these two paramcteis or factors and that only one 
of the measurements (7) involves some additional factor 

We turn next to the alternative reference frame leprcsented by the unit 
reference vectors, B i and B 2 The factor B x was placed so as to be colhnear 


a ' □ D _ 

J L_— 

__ 

□ _J 



Aa D 1_ 

!□ 

□ 

CU 

□ 

1-1 




Thickness factor 


Tallness factor 


B, 0 


□ □ 


Size factor 


* □ □ □ a □ LI! 

short-thick short-thin and tail-thick 


_I [_J Bipolar factor 

tall-thin 


Figure 2 

with the test vector l h which is in the middle of the range of test vectors. 
All the factor loadings are positive and of appreciable magnitude, so that 
this is a general factor In Figure 2 we have the nine cylindrical forms ai- 
ranged in order according to this factor, from the lowest to the highest In¬ 
spection leveals that the cylinders are arranged in order of size, irrespective 
of shape. We might call this characteristic a "general-size” factor. It can be 
seen m the figure that the cylinders aio in ordci according to longiLudmal- 
scction area This happens to be piopoi tional to the measure actually taken 
for the factor B h namely, the side area We might choose a neighboring 
vector as an index of size, such as the volume vectoi 5 The location of this 
factor m the configuration is not unique 
We turn next to the factor B 2 , according to which the nine eylmdiiral 
forms were arranged in the last low of Figure 2 The plot of Figw e 1 i eveals 
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tins as a bipolar factor, and we therefore look for some characteristic which 
can have either positive or negative values, or we look foi a bipolar factor 
that can be thought of as a composite For negative values of the factor R 2 
we find the short-thick cylinders, while at the positive end of the-distnbu- 
tion we find the tall-thm cylmdeis This might be called by a single name, 
such as a “slenderness” factor Negative values for this factor would refer 
to short and thick cylinders In the middle of the distnbution, where the 
factor loadings aie near zeio, we should then expect to find those cylinders 
which are neither tall-thm nor short-thick Here we find those that can be 
classified as short-thin and as tall-thick The two descnptions, "short-thin” 
and “tall-thick,” would both have zeio satuiation on the slenderness factor, 
but tall-thick would have positive saturation on the size factor For some 
purposes of classification or interpretation it might be meaningful to de- 
scube these objects in teims of their general size, such as volume, and theii 
degree of slenderness, but for most purposes this classification would prob¬ 
ably seem artificial In most situations in which a configuration lepresents 
coi relations that are positive 01 zero, we should piobably find the simplest 
explanatory concepts to be repiesented by factorial axes that boidei 01 in¬ 
close the configuration rather than those in which a dominant factoi is an 
average of all the measuies of the battery by going through the middle of the 
configuiation As fai as adequacy of description is concerned, any reference 
frame might be used which can gain acceptance as to the interpretation of 
the factors All three of the frames that have been discussed here are equal¬ 
ly accurate in the description of the correlation coefficients 
We have seen that the refeience frame that is found in the factoring is 
ordinal lly arbitrary We have considered two alternative sets of orthogonal 
reference factors, both of which can be given some physical interpretation 
The location of a meaningful reference frame is not unique, but there some¬ 
times exists a pieferred reference frame that is moie readily inteipieted 
than the others In the nature of the case one cannot prove that the frame 
chosen is in any sense the correct one. Nor can one demonstrate that some 
other frame would not also be meaningful The best that one can do is to 
discover a set of reference factois that can be given plausible interpretation 
to aid in undei standing the underlying ordei among the variables m the 
battery Such findings can be followed by other experiments in the explora¬ 
tion of the factors that are revealed by the factorial analysis It should be 
recognized that a factorial study is successful to the extent that meaningful 
factois can be identified by a proper choice of leference axes 



CHAPTER VII 
THE SPHERICAL MODEL 


The vector model 

In previous chapteis we have considered factoi problems in two dimen¬ 
sions The same concepts will now be extended to problems of three common 
factors The coirelation matrix can be repiesented by a set of vectors whose 
scalar pioducts aie the con elation coefficients. If the rank of the correla¬ 
tion matrix is 3, then theie must be postulated three common factoi s The 



Figure 1 

vector model for such a coirelation matrix would extend to three dimen¬ 
sions, In Figure 1 we have such a model, in which the test vectors aie lep- 
resented by long pins or wires that are stuck into a central cork Each of the 
pms represents a test vector The coirelation between any two tests is 
equal to hih^ cos <£ 12 , where /n and hi are the lengths of the pms and <j>u is 
the angle between them If such a model were given to us, we should be able 
to reconstruct the correlation matrix fiom which the model was built. No 
information is lost in going from the correlation matrix to the model or vice 
versa They give the same information 
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In the vector model of the figure we see that the entire configmation of 
test vectois spans not more than a right angle This means that the inter- 
couelations are all positive If two vectois m the model were at right angles, 
the corresponding conelation coefficient would be zeio The scalar pioduct 
of a test vector with itself is h\ s the angle <pu being zero, and cos <£ n , unity 
Hence the squaie of the length of a test vector j is equal to its communality 

h) 

In Figure 1 the test vectors aie arranged more 01 less at random, except 
for the restriction that theie aie no obtuse angular relations. Such a con- 



Fiqure 2 

figuration would be pioduced if we were to write a correlation matux at 
landom with positive or zero coefficients and subject to the restriction that 
it be of rank 3 to correspond with the three dimensions of the model In 
Figure 2 we have a configuration of test vectors showing thi ee clusters The 
tests within a clustei would have high mtercorielations, while pairs of tests 
from different clusters would have low conelations If the tlnee clusters 
were at light angles, the tests from different clusteis would have correla¬ 
tions of zero or near zero If such a configuration were found, it would be 
natural to suspect that the tests represented three, different kinds of traits 
winch are statistically independent The clusters would be apparent by in¬ 
spection of the correlation matrix because the tests could be so grouped 
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that the tests within a given group would be highly correlated, while tests 
from different groups would show low correlations 

Figure 3 shows a different type of configuration, m that the test vectors 
are arranged m three planes The configuration could be descubed as a 
three-sided cone with test vectois m each side of the cone and with apex at 
the centei of the coilc If such a configuration of test vectois were found, it, 
would be natuial to suspect that thiee parameters or factois had detemuned 
the correlations and that the paiarneters could be repicscnted in the simplest 
way at the corners of the configuration, Each test vector that lies in one of 


* 



the co-ordinate planes could then be descubed as a linear combination of 
two primary 01 co-ordinate vectors 

The same kind of configuiation with the test vectois in the tlnee sides 
of a cone could also be lepresented in the model of Figure 4, m which the 
thiee sides are represented by caidboards on which have been drawn the 
vectors of the test configuration 

The spherical model 

1 hree-dimensional test configuiations can be lepresented by models of 
the kind that have been lllustiated, but it has been found moie convenient 
to represent them on the surface of a sphere, as shown in Figure 6. In ol¬ 
der to lepresent a test conhguiation on the suifaec of a spheie, the test 
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vectors are first extended to unit length The termini of the vectors can 
then be represented as points on the surface of a unit sphere For the pur¬ 
pose of showing a test configuration graphically in three dimensions, one 
can use to advantage a blackboard sphere of the kind used m teaching spher¬ 
ical trigonometry On the sphere of Figure 6 we have the same type of test 
configuration as in Figure 2, where the tests are arranged m three clusters 
In Figure 6 we have the same type of configui ation as in the vector model of 
Figure S, where the test vectois he in three planes, 

If we should find that the configui ation of a test batteiy looked like Fig¬ 
ure 6, then the simplest desciiption of the test vectors would be in terms of 



Figuee 4 

the vectors A, B, and C at the corners of the configuration All the tests on 
the side AB could be descnbed m terms of the two vectors A and B, the 
test vectors on the side BC could be described in. terms of the vectors B and 
C, and similarly for the test vectois on the side AC in terms of the vectors 
A and C We should then make the hypothesis that the vectors A, B and C 
at the corners of the configui ation represent underlying properties or sim- 
p l ymg parameters m teims of which the conelations of the whole test bat- 
ery can be understood A test vector near the middle of the side AB would 
then be described as a linear combination of A and B with equal weights 
or A and B A test vector near A on the side AB would also be described 
as a linear combination of A and B, but the weight would be relatively larger 
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for A and smaller foi B In a configuration of this kind each test vector 
could be described in terms of less than three factors, even though the bat¬ 
tery as a whole required three factors. This is the principal chai acteristic 
of a simple structure, which is the combination of a test configuration and a 
set of co-oidinate axes m which each test vector lies in one or more of the 
co-ordinate planes, as in Figure 6 If there should be a test m the battery 
that involved all three of the factois A, B, and C, then it would lie inside 
the spherical triangle, and it would not contribute towaid the location of 
the co-ordinate planes. 

In Figure 7 we have a model of test vectors in three dimensions in which 
the triangular configuration is incomplete Here we have no tests which con- 
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tain only the common factoi A or the common factoi C The tests neai B 
contain only one of the three common factors, namely, B Furthermore, we 
see that all the tests m the battery have some saturation on the factor B 
because the tests on the side AB are linear combinations of A and B, while 
the tests on the side BC are linear combinations of B and C Hence the 
factor B would here be a general common factor which is present in all the 
tests of the batteiy In a configuration of this kind the location of the plane 
AO is indeterminate, and so are also the co-ordinate vectors A and C as 
tar as the configuration is concerned The only vector that is heie deter¬ 
minate is the geneial common factor B * 


that r be °° nfU8ed Wlth the kmd of ^cral common factor 

confiiu aUon T, Q by locatlng the facto1 - » center of the 

nnguration as a whole Such an axis will neressanly be inside the spherical triangle, 
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Plotting the spherical configuration 

In order to study a configuration of test vectors in thiee dimensions, it is 
useful to extend them to unit length so that the configuration can be easily 
plotted on the surface of a spheie When that has been done, we can see the 
whole three-dimensional configuration, as shown m the examples of Figuies 
B, 6 7, 10, and 11 The process of extending the test vectors to unit length 
will be called normalizing the lest vectoi s 

In Table 1 we have a given factor matux of twenty measures or tests and 
three oithogonal factors, I, II, and III The first stop m noimalizmg the 

Table 1 

Normalizing of Test Vcolors 


Factor Matrix 
P 


Normalized Factor 
Ma trix 

I" n 


Test 

r 

II 

III 




/; 


i/’ll, 

r 


rr 

rir 

i 

659 

- 736 

138 


9950 


997497 

1 

002509 

661 

— 

738 

138 

2 

725 

180 

- 656 


9884 


994183 

1 

005851 

729 


181 

- 660 

3 

665 

537 

500 


9800 


990252 

1 

009844 

672 


542 

505 

4 

869 

- 209 

- 443 


9951 


907547 

1 

002459 

871 

— 

210 

- 444 

5 

834 

182 

508 


9867 


993328 

1 

006717 

840 


183 

511 

6 

.836 

519 

152 


9914 


995091 

1 

004328 

840 


521 

153 

7 

856 

- 452 

- 269 

1 

0094 

1 

004670 


995691 

852 

— 

450 

- 268 

8 

848 

- 420 

320 

1 

0030 

1 

001500 


998502 

847 

— 

425 

320 

9 

861 

416 

- 299 

1 

0038 

1 

001898 


998106 

859 


415 

- 298 

10 

880 

- 341 

- 354 

1 

0160 

1 

007968 


992095 

873 

— 

338 

- 351 

11 

889 

- 147 

430 

1 

0020 

1 

001000 


999001 

888 

— 

147 

.436 

12 

875 

485 

- 093 

1 

0095 

1 

004788 


995235 

871 


483 

- 093 

13 

067 

- 725 

109 


9824 


991161 

1 

008918 

673 

— 

731 

110 

14 

717 

246 

- 619 


9578 


978673 

1 

021792 

733 


251 

- 632 

15 

634 

501 

522 


9254 


961977 

1 

039526 

659 


521 

543 

16 

936 

257 

165 


9694 


984581 

1 

015660 

951 


261 

168 

17 

966 

- 239 

- 083 


9972 


998599 

1 

001403 

967 

— 

239 

- 083 

18 

625 

- 720 

166 


9366 


967781 

1 

033292 

646 

— 

744 

172 

19 

702 

112 

- 650 


9278 


963224 

1 

038180 

729 


116 

- 675 

20 

664 

536 

488 


9663 


983006 

1 

017288 

675 


545 

496 


vectors is to deteinnnc the communalities h? as shown The length of each 
vector h, is then detei mined, as well as its iceiprocal \/h, = d Jt which be¬ 
comes a multiplier for each low of the given factor matux The augmented 
factor loadings A lm of the normalized factor matrix are shown in the last 
three columns of Table 1 The data are then ready to be plotted on a sphere 
The thiee entnes in any iow of the normalized factoi matrix F n give the 
projections of the normalized test vector J on the thiee oithogonal axes 


and its location will be determined entirely by the particular collection of tests that hap¬ 
pens to be used as a battery in the factor analysis This pioblem is discussed in later 
chapters 
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Each projection A, m is the cosine of the angle between the normalized test 
vector J and the refeience vector M The relation between the given factor 
matrix F and the normalized factor matrix F n can be stated formally in the 
matrix equation 

(1) F)F — F n , 

where D is a diagonal matrix with the stretching factoi dj, or, by the ele¬ 
ments, 

(2") djQijm Ajm i 

Before plotting the spherical configuration, we fust draw on the spheie 
three mutually orthogonal great circles If the sphere is to be used repeated¬ 



ly for factorial problems, it is best to paint or scratch the three gieat, circles, 
so that they will not be erased for each problem One of the octants is 
chosen as positive for all three co-ordinate axes I, II, III, as shown in 
Figure 10, and the thiee co-ordinate axes are labeled. 

In plotting the test vector termini on the surface of the sphere it is con¬ 
venient to use a strip of paper of length about twice the diameter of the 
sphere On this strip are marked the surface distances corresponding to the 
cosines A, m In Figure 8 we have represented the strip S, which has also 
been drawn in Figure 9. The point 1 00 on the scale is placed at the refer¬ 
ence axis I on the sphere The point on the scale at axis II is marked zero 
The surface distance from I to II is then 


( 3 ) 


^ = 0 7584 d = t 
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The intermediate points on the scale aie labeled directly in terms of the 
corresponding cosines, which are the entries in the normalized factor ma¬ 
trix F n . For the projection A Jt in F n we have the corresponding angle 

(4) 4>,i = cos-lAjj 

and the corresponding suiface distance 


(5) 


pt 


t cos^Aji 

90 


For any given sphere the scale will be pioportional to that of Figure 9 The 
measuring strip should be made twice as long as the scale indicated in Fig- 


too BO 97 95 

90 

60 

70 

60 50 40 30 20 tO 0 

r i i ni _ I _ i _ i i i i ri i i m m i i i i i 
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ure 9, so that measurements can be made of both positive and negative 
cosines. 

In locating the terminus of a test vector on the sphei e, we find a point 
which has the required surface distances fiom the three orthogonal axes 
The third measurement checks the location inferred from the fiist two 

Table 2 


Values of p for Given Values of A Jm = cos <l jin 


a 

* 

a 

P 

a 

P 

a 

P 

a 

P 

1 00 

00 

95 

20 

70 

61 

45 

70 

20 

87 

99 

09 

90 

29 

65 

55 

40 

74 

15 

.90 

98 

13 

85 

35 

60 

59 

.35 

77 

10 

94 

97 

16 

80 

41 

55 

63 

30 

81 

05 

97 

.96 

18 

75 

46 

50 

67 

25 

84 

00 

1 00 


measurements When the scale has been prepaied on a strip of papei or 
cardboard, any three-dimensional configuration is readily plotted on the 
sphere, so that the whole configuration can be seen directly 

In Table 2 we have listed the values of p for different given values of the 
normalized factor loadings A jm These values of p can be used for making a 
scale to fit a sphere of any given diameter d. 

The piesent battery of twenty measuies or tests has been plotted on the 
sphere of Figure 10, Each point has been numbered to identify the corre¬ 
sponding test vector The three great circles are indicated, as well as the 
three co-ordinate axes I, II, and III Inspection of the configuration on the 
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sphere shows that the points are arranged in the form of a triangle and that 
all but two of the test vectois lie m the sides of the spherical tuangle In¬ 
stead of using the aibitiary lefeience frame I, II, III, which was obtained 
by the factoring method, we draw a new refeience frame, as shown m Fig¬ 
ure 11 By using these new refeience axes, wc should be able to describe 
each of the test vectors that lie m the sides of the sphencal triangle in teims 
of one or two lefeience axes instead of thiee, as requned in the frame of 
Figure 10 The three gieat circles are so drawn as to pass thiough or near 
groups of vectors, as shown in Figure 11 The new refeience vectois are the 
normals to the thiee great circles, and the pnmaiy factors are determined 
by the three intersections of the new gieat circles 



Figure 10 


With the model as shown m Figuie 11 we aie leady to put the Lie tonal 
solution mto numerical form Considei the great wide diawn through or 
neai the points 19, 2, U, 9, 12, 6, 20, S, 16 By means of the measuimg 
s .up locate a point on the suiface of the spheie which is orthogonal to this 
cncle It represents one of the unit lefercncc vectois, which may be denoted 
In t le S£Lwe manner locate the other two refeience vectois oithogonal 
to the other two planes One of the reference vectois is shown m Figure 11 
namely, A z ’ 


By means of the measuring stnp, measure the thiee diiection cosines for 
each of the leference vectors and record them in columns of the transforma¬ 
tion matnx A, as shown m Table S In determining the duection cosines of 
a ieference vector, measure its suiface distance from each of the thi ee orthog¬ 
onal axes I, II, III, on the sphere In the first section of the table we have 
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the given factor matrix F, and m the last section we have the desned factor 
matrix V, which describes the twenty test vectors in terms of the three new 
reference vectois A„, where the subscript p refers to the new axes When the 
measurements have been ta,ken from the sphere, it is advisable to adjust 
them m the second or third decimal, so that the sum of then squaies is 
equal to unity This step insures that each of the refeience vectors is of 
unit length. 

When the transformation matrix A has been wntten as in Table 3, we 
can cany out the matrix multiplication 

(6) FA = V . 



The entries in this matux show the projection of each test vector on each 
of the thiee new leference vectois The point SO, foi example, is on oi neai 
two of the new co-oidinate planes, as shown in Figure 11, and hence it 
should have two factoi loadings that aie zcio oi neai zeio, as shown m 
Table 3 

The new leference vectois are not orthogonal The cosines of thcii angu- 
lai separations can be deteimined by the measuring strip and recoided in 
the matrix C of Table 3 The numeiical values of these cosines can also be 
deteimmed without direct measurement by the equation 

(7) A'A = C , 

or, in terms of the elements, 
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where the subscripts p and q refer to the new axes The transformation 
matrix A is not an orthogonal matrix, because the three new refeienee vec¬ 
tors are not orthogonal The columns are normalized to coi respond to the 
fact that the reference vectors are unit vectors The cross products of pairs 
of columns of the transformation matrix A give the cosines of their angular 
separations, as m equations (7) and (8). In an orthogonal matrix these co- 

Table 3 

The Transformation Matrix A 


Given Factor Matrix F 


Transformation A Rotated Factor Matrix V 


Test 

I 

II 

III 


A* 

Ay 

A* 

Test 

Aa- 

Ay 

A, 

1 

659 

- 736 

138 

I 

483 

466 

479 

1 

969 

003 

— 003 

2 

725 

180 

- 656 

II 

- 834 

254 

560 

2 

025 

939 

005 

3 

665 

537 

6U0 

III 

267 

- 847 

675 

3 

007 

023 

957 

4 

.869 

- 209 

- 443 





4 

476 

727 

000 

5 

.834 

182 

608 





5 

387 

005 

844 

6 

836 

.519 

152 





6 

012 

393 

704 

7 

856 

-.452 

- 269 





7 

.719 

512 

— 025 

8 

.848 

- 420 

320 





8 

850 

016 

384 

9 

861 

416 

- 299 





9 

- on 

760 

444 

10 

.880 

- 341 

- 354 





10 

615 

023 

— OQS 

11 

.889 

- 147 

436 





11 

668 

008 

638 

12 

875 

485 

- 093 





12 

- 007 

610 

628 

13 

667 

- 725 

109 





13 

956 

034 

— 013 

14 

717 

246 

- 619 





14 

- 024 

921 

.063 

15 

634 

501 

522 





15 

028 - 

019 

.037 

16 

936 

257 

165 





16 

282 

362 

704 

17 

966 

- 239 

- 083 





17 

644 

460 

273 

18 

625 

-.720 

166 





18 

947 - 

032 

008 

19 

702 

.112 

- 650 





19 

072 

.906 

— 040 

20 

664 

536 

488 





20 

004 

032 

948 


Primary Vector T 


T , 

661 

- 736 

144 

Tv 

[ 713 

199 

- 671 

T. 

654 

545 

524 


TA. = D 


T, 

\ 972 

0 

0 

Tv 

0 

951 

0 

T, 

0 

0 

972 


Correlation of Primaries Cosines of Reference Vectors 



T , 

r v 

T, 

T, 

1 000 

226 

105 

T v 

229 

1 000 

224 

T, 

105 

224 1 000 


TT'=R„ 



Ax 

Ay 

A, 


1 000 

- 213 

- 055 

Ay 

- 213 

1 000 

- 206 

Ai 

- 055 

- 206 

1 000 


A'A = C 
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sines aie zero because such a transformation represents the rotation of a 
rigid orthogonal reference frame 

The primary vectors which lepresent the thiee underlying factors or 
parameters in pure foim are unit vectors at the corners of the configui ation 
A unit vector at the cornei near the test vectors 3, 15, and SO repi esents one 
of these primal y factors It is defined by the intersection of two of the co- 
oidmate planes Let the primary test vectois be denoted T x> T y , and T x 
The direction cosines of each of these unit vectors may be determined by 
the measunng strip and lecoided in the rows of the factoi matrix, as shown 
in Table 3 Here we are treating the three primary vectois as if they were 
additions to the given factor matrix If we had three perfect tests of these 
pnmai y factors with no unique variance, they would appear m the factoi 
matrix, as shown m the rows T x , T v , and T z 

If we add the three pumary vectors to the given factoi matrix and cany 
out the tiansformation, 

(9) FA = 7 , 

for the last three rows of the factor matnx, we have the corresponding 
equation, 

(10) TA — D , 

where the product must be a diagonal matrix, as shown in Table 3 By defi¬ 
nition each primary vector lies in two of the co-ordinate planes, and hence 
it must have zero projections on two of the reference axes The result is a 
diagonal matrix If the reference vectors were orthogonal, they would coin¬ 
cide with the primary vectors, but, when the reference vectors are oblique, 
as in the present case, the product is a diagonal matnx, m which the diago¬ 
nal entries deviate from unity The diagonal entries are, in fact, the cosines 
of the angles between the primary vectors T p and the corresponding refer¬ 
ence vectors A p , as can be verified on the sphencal model 

The matrix T, which shows the direction cosines of the pnmaiy vectors, 
can be determined from the transformation matrix A without direct meas¬ 
urement on the sphere. By (10) we have 

(11) T = DA- 1 

We find the inverse of the transformation matrix A and normalize the rows 
of that matnx Normalizing the rows of A -1 is represented in equation 
(11) by the premultiplication with the diagonal matrix D The result is the 
matrix T, which shows the direction cosines of the primary vectors. 

The correlations between the primary factors can be determined by di¬ 
rect measurement on the surface of the sphere The same correlations can 
be determined from the transformation matrix A without direct measure- 
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meat Just as the correlations between the tests are given by the funda¬ 
mental factor theorem, 

(12) FF' = R,k — R, 

so for the last three rows of the factor matrix we also have the coi respond¬ 
ing equation, 

(13) TT' = R m 
Substituting (11) m (13), we obtain 

(14) R rv = (DA' 1 ) (DA" 1 )' 
or 

(15) R m = D(A'A)" ! D 

By this equation the conelations between the primary factors can be deter¬ 
mined When the product (A'A)" 1 has been computed, the diagonal matrix 
D is so written that the diagonal entries of the coi relation matux R pa are 
unity, since the scalar pioduet of a unit vectoi and itself is, of course, unity. 
The diagonal matrix m (15) is the same as the diagonal matrix shown in 
m the lower section of the oblique factor matrix V Each entry in D is the 
cosine of the angle between a primary vector T p and the corresponding 
leference vector A p 

For some purposes it is useful to designate explicitly the general ele¬ 
ments of the several matrices of Table 3 In the given orthogonal factor 
matux we have the elements a jm , and m the transformation matrix we have 
Xm,i, wheie the subscript p refers to primaiy factors The product is 

(lb) a, m — Vjp , 

where the elements v, p of V show the projections of the test vectois j on the 
lefeience vectors A p The elements of the matux T are denoted t pm The em¬ 
ulation matiix for the pumary factors is denoted R pa , to distinguish itfiom 
the correlation matux for the tests, which is written R lh , or simply R with¬ 
out subscript Let the elements of (A'A)" 1 be denoted b pq Then the diagonal 
elements are b pp The diagonal elements of D are, then, d PI or 



The present example has been discussed m detail for the three-dimension 
al case to show T the relation between the computational work and the geo¬ 
metrical interpretation In the interpretation of primary factors we inspect 
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each column of the oblique factor matrix V to discover, if possible, the na¬ 
ture of the trait which is present in the tests with significant factoi loadings 
and absent m the tests with negligible loadings on the factor There is no 
guarantee beforehand that a simple configuration like Figure 11 can be 
found in any given problem, and there is no guarantee that a clarifying de¬ 
scription of the factois can be obtained That is detcimined by the intui¬ 
tions of the mvestigatoi in setting up lus faetonal experiments so that 
meaningful and significant results can be obtained 

Correlated factors 

In the previous discussion, examples of both oithogonal and oblique ref¬ 
erence vectors have been given A set of orthogonal leference vectors is a 
geometiical lepiesentation of factois which are uncorrelated, and coire- 
lated factois aie shown by an oblique reference fiame Among statisticians 
and psychologists there is a lather general belief that if human traits are to 
be accounted for by any kind of factois, then these factois must be uncoire- 
lated This belief has its ongin m the statistical and mathematical conven¬ 
ience of uncorrelated factors and also in our ignorance of the natuie of the 
underlying structure of mental tiaits The reason for using uncon elated 
reference traits can be undoistood, but it cannot be justified Height and 
weight are two useful measuies of body size, even though they aie cone- 
lated. If we should use, instead, two lineai combinations of height and 
weight which are uncoiielatecl in the geneial population, we might find such 
measures moie convenient in statistical theory, m that the cross products 
would vanish and we could find a certain satisfaction in that the two meas¬ 
ures are independent, but they would be awlcwaid to think about 

We might as well make use of the freedom of factoi analysis as legards 
orthogonality by allowing the primaiy factois 01 parameters to be corie- 
lated In the physical and biological sciences we have the exact counterpait 
of this situation There we deal with parameters that usually have some 
physical meaning, such as momentum, acceleration, blood piessuie, the 
limmal stimulus, and thousands of othei parameteis that have meaning in 
conceptual representation Scientists seldom wony about the fact that 
parameteis are con elated in a random group of people who might be the 
subjects in a scientific study, unless such covanance is itself the object of 
study in lelation to underlying theory If two paiametcis should be found 
to he nearly perfectly con elated, the fact would suiely be noticed, and we 
would then try to reduce the corresponding concepts to an identity by re- 
foimulating fundamental ideas about the phenomena 

The restriction that parameters be uncon elated appeals only in those in¬ 
vestigations in which we start with no ideas The con elation coefficient is 
a symbol of complete ignorance It is i ai ely, if ever, used in those problems 
which are well understood Let us take an example to show how unneces- 
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sary it would be to insist on zero correlation between parameters when the 
relations aie well understood Let us suppose that nothing is known about 
the nature of rectangles except that we can take measurements on them 
We might measure five hundred rectangles and discover that there is a corre¬ 
lation of + 70 between the lengths and the widths in our collection of rec¬ 
tangles Such a finding would be determined by the fact that big rectangles 
tend to have long sides and to be wider than small rectangles We might 
have the conviction that the piineipal piopeities of rectangles could be de¬ 
termined from their lengths and widths, but we should have to meet the 
restriction that the two parameters must be uncorrelated m the particular 
set of rectangles that we collected for measurement That could be done if, 
for instance, we took the length as one measurement and, say, (W— JL) as 
the other measurement, where L and W are the length and width, respective¬ 
ly. The linear combination would depend on the particular set of rectangles 
in the experiment. Two measures could easily be found which were uncoi- 
related in our particular collection of rectangles, but the result would not 
be very useful m thinking about rectangles As another example, m taking 
physical measurements of men, it is customary to record height and weight 
If we were forbidden to use these two measures because they are con elated, 
the absurdity of such a restriction would be self-evident On the other hand, 
it should be recognized that if we deal with a group of measures that are 
highly correlated, we are likely to be confining ourselves to one domain, and 
it is then useful to look for other significant measures that cover additional 
features of the things that are being measured In general, measures that 
cover different aspects of an individual are likely to show lower correlation 
than those which represent the same domain, but it does not follow that 
our fundamental concepts should be in any sense restricted to those which 
show zero correlation in a random group of people In developing the fac¬ 
torial methods we have insisted that the methods must not impose ortho¬ 
gonality on the fundamental parameters that are chosen for factorial de¬ 
scription, even though the equations are thereby simplified in that the cross 
products vanish In the methods that we have developed, each author is 
still free to impose orthogonality on the factors if he so chooses or if the 
nature of lus problem is such that this restriction is indicated. 

The box problem 

The box problem is an example involving three parameters or factors 
that are represented in the spherical model of Figure 11 * A physical ex- 

* The numerical problem that is used in this ohnpter is the so-called “box problem," 
which was designed to coriect some current misconceptions about factor analysis Sec¬ 
tions from the original publication of the box problem will be used here to illustrate by a 
physical example the interpretation of primary factors as simplifying parameters that 
are not necessarily unique (see “Current Issues m Factor Analysis," Psychological Bulle¬ 
tin, XXXVII, No 4 [April, 1940], 189-236) 
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ample of two parameters or factors—the cylinder problem—was used in 
chapter in. 

A random collection of boxes constitutes the population in this example. 
Let us imagine a collection of boxes and a set of twenty measurements for 
each box Let us assume, as is really the case in most psychological problems, 
that we have no idea what these measurements represent and that we believe 
some underlying order may exist in the twenty measurements for each box 
In this fictitious example the boxes constitute the individuals in a statistical 
population, and the measurements correspond to twenty test scores for each 
individual In Table 4 we have the three dimensions, x, y, z, for each box 


Table 4 


Boxes 

Dimensions 

X 

y 

3 

i 

3 

2 

1 

2 

3 

2 

2 

3 

3 

3 

1 

4 

3 

3 

2 

6 

3 

3 

3 

6 

4 

2 

1 

7 

4 

2 

2 

8 

4 

3 

1 

9 

4 

3 

2 

10 

4 

3 

3 

11 

4 

4 

1 

12 

4 

4 

2 

13 

4 

4 

3 

14 

5 

2 

1 

15 

5 

2 

2 

16 

5 

3 

2 

17 

5 

3 

3 

18 

5 

4 

1 

19 

5 

4 

2 

20 

5 

4 

3 


The box population is here defined in terms of twenty different rectangular 
box shapes, and it is assumed that these twenty shapes occur with equal fie- 
quency in the population of boxes For example, if the population consists 
of two hundred boxes, there are ten of each kind Any other arrangement 
could have been taken In Table 6 we have a list of the twenty measure¬ 
ments (tests) that were taken for each box. These measurements consist of 
various non-linear functions of the three dimensions, x, y, z, but in the factoi 
analysis the number of basic factors and their combination in the twenty 
measurements are assumed to be entirely unknown 
It will be seen m the formulae of Table 5 that each of the tests 1, 2, and 3 
represents the square of an edge of the box; 4, 5, and 6 represent the area of 
aside; 7, 8, and 9 repiesent the diagonal of a side; 10, 11, and 12 repiesent 
the perimeter of a side, and the other tests are similar arbitrary functions 
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of the tlnee box mcsiHiuements In dealing with the twenty measuiements 
foi each box', vve tieat them meirly as if they weie twenty test scores for 
each box We con elate them and factor them as if wc knew nothing about 
the mannei m which the scoies became conelated Oui object is to discover 
by factor analysis whether 01 not the twenty measurements for each box 
leveal some underlying physical order Actually, we have set up the meas¬ 
urements so that tlnee factois aie involved—namely, the thxee dimensions 
of a box—-but we deal with the problem factonally as if we did not even 
know how many factois were involved m the twenty test scores foi each 


Table 5 


Tests 

Stiuctuic 

X y s 

Test Formulae 

i 

+ 



X 2 

2 


+ 


y- 

3 



+ 


4 

+ 

+ 


xy 

5 

+ 


+ 

xz 

6 


+ 

+ 

y* 

7 

+ 

+ 


Vx*+v l 

8 

4- 


+ 

Va -?+z* 

9 


+ 

+ 


10 

+ 

+ 


2x+2 y 

11 

+ 


+ 

2x+2i 

12 


+ 

+ 

2y+2z 

13 

+ 



log * 

14 


+ 


logy 

15 



+ 

log s 

16 

+ 

+ 

+ 

xyz 

17 

+ 

+ 

+ 

Vx a 4-2/ 2 H-2 2 

18 

4 * 



«•* 

19 


+ 



20 



+ 

e z 


box In oi dei to simplify the problem, eiiois of measuiement and sampling 
enois are heie omitted The correlations aie theiefore unaffected by the 
number of multiples of the twenty different box shapes that constitute the 
statistical box population. The coirelations were determined from the box 
measurements listed in Table 4 and the test formulae of Table 5 In Table 3 
we have the centroid matrix for three factors The mean of the absolute 
residuals (disregarding sign) after thiee factois had been extracted was 
008, which does not wanant the extiaction of more factois At this point 
we would know that only thi ee factors weie involved m the twenty fictitious 
box scores 

In the interpretation of the rotated factor matrix V of Table 3 we look for 
the tests which have the high saturations m the fiist column These are 
tests 1, 13, and 18, and we conclude that the first primaiy factor is veiy 



THE SPHERICAL MODEL 


143 


heavily lepiesented m these tests In this fictitious pioblem we can actually 
turn back to Table 5 to find the formulae for those tests in teims of the pn- 
maiy factois These are, of couise, not known in psychological problems 
We find that tests 1, 13, and 18 are the only tluee tests which depend ex¬ 
clusively on the ^-dimension of the boxes 

In the thud column of the factoi matnx V of Table 3 we find throe tests 
with high saturations, namely, 3, 15, 20 These tlneo tests aie theiefoie 
judged to have most of then variance on one of the undeilymg factors In 
Figure 11 we find these three tests at one corner of the configuration, and we 
conclude, therefoie, that they involve only one of the three common factors, 
whatevei they may be Turning back to Table 5, as we can do in this ficti¬ 
tious example, we see that the three test scores are the only ones that are 
functions of the parameter z 

Considei the bottom low of tests in Figuie 11 These tests aie 1!), 2, 14, 
9, 12, 6, 20, 3, 15 Since these test vectois lie in one of the co-ordinate 
planes, we mfei that theie is one factor that is absent m all these tests and 
that the factor is conspicuous in 1, 13, and 18, which lie at the opposite 
coiner of the configuration Turning to Table 5, we see that this factor is x 

The same kind of compaiison can be made for each of the thiee pnmaiy 
factois x, y, and z Those tests which depend on only one of the pi imai y 
factois aie in the conesponding cornei of the diagiam, and they aie lepie- 
sented in the factoi matrix V with laige factor loadings in the piopei col¬ 
umn Those tests which ai e functions of two pi imai y factoi s arc l cpi esented 
as points on the sides of the triangle, and the points are consistent tlnough- 
out as to the pans of pnmaiy factois involved Finally, the two tests 16 and 
17, which have formulae containing all throe of the pnmaiy factois, arc in¬ 
side the triangle They are the volume and the pnncipal diagonal, lespec- 
tively It is interesting that the linear approximations which are assumed 
in factor analysis reveal a simple configui ation, even though the twenty 
measurements weie nonlinear functions of the primaries The disciepancies 
aie lepresented by the small entries m the factoi matnx V, which would be 
zero if the factoi methods did not depend on the assumption of hiicauty as 
a fust approximation 

Next we note that the tluee pnmary factors in this population of boxes 
aie coirelated The correlations among the pnmaiy factois as deteimmed 
by the factoi analysis aie shown in Table 3 The actual coi i elutions between 
the piimanes m the original data aie shown in Table 11, and the agreement is 
self-evident In setting up this example we assumed that a landom collec¬ 
tion of boxes would show correlation between the three dimensions A box 
that is tall is likely to be thick and wide, probably to a greatei extent than 
the conelations which we assumed m the fictitious problem 

It would be possible to set up an example with twenty measuiements 
so that every measuiement would involve all three of the box dimensions 
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A factor analysis of such a system of measurements would not reveal a sim¬ 
ple structure, so that no clear bounding planes could be determined In 
psychological experiments we must recognize this possibility Some factor 
experiments will fail to reveal primary factors This will necessanly happen 
if each variable involves as many factors as there are factors in the whole 
set of variables There is no guarantee beforehand that this will not happen. 
On the other hand, it is extremely unlikely that a simple structure will ap¬ 
pear with the same identifiable primaries in several independent factor anal¬ 
yses of different test batteries that are given to different populations, unless 
the structure is valid 

The interpretation of the primary factors m the factor matrix V would 
consist in studying the several test variables that show high saturation on 

TabU 6 


Correlation Matrix 



X 

y 

z 

X 

1 00 



Y 

25 

1 00 


z 

10 

25 

1 00 


each primary factoi. One would tiy to find what it is that is common to 
these variables or measures, One might discover that, in general, tallness is 
a common characteristic of one primary and that the best measurements 
available for tallness are those which show highest saturation on the factor 
We might name it a “tallness” factor. That would merely imply that the 
twenty measurements are functions of three linearly independent parameters 
and that one of them is tentatively named "tallness " A box which meas¬ 
ures high on that factor would stand highei from the floor than boxes that 
are low in that factor Now this identification of the linearly independent 
parameters in the twenty box measurements would not imply that the tall¬ 
ness factor has some sort of locus in each box or that the factor is a separate 
organ or gadget in each box or that it is some sort of isolated, abstract, sta¬ 
tistical box-element that has nothing in common with the rest of the box 
It is an independent parameter that is involved m the box measurements 
Exactly the same procedure is followed m dealing with psychological data. 
The identification of a primary factor in psychological test experiments is 
merely a challenge for us to find out what sort of process or attribute it is 
in terms of known concepts or else to invent new concepts that, fit the ex¬ 
perimentally determined primary categories 

The attempts to interpret the primary factors have often been criticized 
because they are not sufficiently objective Let us be clear about two dis¬ 
tinct problems that are involved here It is one problem to isolate a primary 
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factor and to determine by repeated experiments that it has some functional 
uniqueness That is factor analysis proper It is another problem to find 
the psychological or physiological meaning of a functional uniqueness when 
it has been determined. That is a matter of interpretation, and consequent¬ 
ly it is necessarily subject to debate, with conflicting interpretations. We 
must remember that the interpretation of every scientific experiment is sub¬ 
jective. There is no kind of scientific experiment m which the interpretation 
rolls out “objectively.” If there is such a factor as auditory imagery, for ex¬ 
ample, just how should the auditory character of the factor produce itself 
objectively m the statistical analysis and without the subjective interpreta¬ 
tion of the investigator 9 Or just how could tallness in the box population ap¬ 
pear objectively in the data? Factor analysis is no exception to other kinds 
of scientific experimentation It is a fortunate circumstance, however, that 
different interpretations of a primary factor can usually be resolved as ques¬ 
tions of fact New factorial experiments can be made to determine which 
interpretation is the more plausible 

In naming the primary factors it is a bettei policy to name the factors in 
terms of well-known concepts, such as Number, Space, Verbal, and Memoiy 
factors than to name them m some noncommittal way, such as x h x 2 , and 
so on If we name a factor “Number,” it will provoke experimentation with 
numbei tests and with nonnumencal tests, and the experiments arc likely 
to be made in terms of psychological hypotheses that can be sustained or 
disproved expenmentally In this way we shall advance faster than if the 
primary factors are left as interesting statistical cuiiosities They should be 
recognized as psychologically challenging experimental effects We cannot 
expect to be coriect in all interpretations of primary factois, and we shall 
probably have occasion to revise interpretations repeatedly as moi e experi¬ 
mental information becomes available. Again, this policy is consistent with 
the desire to make factor analysis a useful psychological tool instead of a 
mere statistical routine for condensing test scoies and correlations. 

Those who criticize factor analysis as faculty psychology talk in the next 
bieath about verbal and nonverbal intelligence, special aptitudes foi music 
and for art, mechanical aptitudes, and disabilities in reading and arith¬ 
metic, without lealizmg that they are implying factor analysis and the inter¬ 
pretation of factors Conversationally, we can talk about musical ability m 
the singular, but, as psychologists, we believe that diffeient abilities are 
probably involved in a good voice, absolute pitch, onginality m harmony, 
counterpoint, orchestration, melodic memory, ease in memoiizing at the 
piano, ability in musical interpretation, and so on How many important 
functional unities might there be m this domain? They are suiely not all 
completely independent They are correlated, and they probably represent 
a smaller number of abilities than the infinite number of musical tasks that 
could be given to a music student To find these functional unities is the 



X 40 MULTIPLE-FACTOR ANALYSIS 

pioblem of factor analysis If that is faculty psychology, then so is most in¬ 
vestigation of individual differences and most of the work on special apti¬ 
tudes and defects And so also are many of the chapter headings in the 
textbooks of psychology which the same critics are teaching 

Correction for uniqueness 

The coirelation between two experimentally obtained measures is some¬ 
times called a "fallible’' coefficient m the sense that the correlation is some¬ 
what depressed by the experimental enovs in the measures If the two 
ineasuies are the best available for two traits x and y and if it is desned to 
estimate the true con elation between the traits themselves as it would be 
expressed by peifect tests of the two traits, then the obtained or fallible co¬ 
efficient is corrected for attenuation The correction for attenuation is an 
estimate of what the eon elation between two tests would be if the error 
variances weie eliminated The estimate is made in terms of experimentally 
determined lehability coefficients for both tests If a factoi matnx is avail¬ 
able showing the loadings on common factors, specific factors, and eiror 
factors, then the correction foi attenuation can be obtained by eliminat¬ 
ing the erioi factor loadings and then normalizing the remaining common 
factoi s and specific factors for each test, so that the sum of their squares 
is equal to unity The con elation between two tests corrected for attenua¬ 
tion would then be the cross pioduct of then respective lows in the normal¬ 
ized factoi matrix foi common and specific factors The same numerical re¬ 
sult is obtained by the usual foimula for this coirection, namely, 


(17) 




wheie i is the augmented coefficient, coirected foi attenuation, and r xx 
and r vv are the reliability coefficients 

The correction for uniqueness is analogous to the collection foi attenua¬ 
tion Instead of eliminating the error variance, we can imagine the test so 
modified as to eliminate both error variance and specific variance The le- 
sult would be a measure containing only the common factors normalized 
so that the sum of the squares of then loadings is unity That is what Ta¬ 
ble 1 represents The correlation between two tests as determined by the 
noimalized common factor matrix is 


(18) 




xy _ ? xy 

V* h% fly hxhy 


m which communahties are substituted foi l eliabilities The correlations 
coirected for uniqueness are the scalar products of pairs of unit test vectors 
as iep 2 ’csented on the spherical model These collections aie useful in some 
theoretical pioblems, and they are represented m the illustrative problem 
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of this chapter by the cosines of the angles between the normalized test vec¬ 
tors In the. present case the principal intei est is in studying the configm a- 
tion of test vcctois and not in estimating what the oonelations would be if 
the tests weie m any way alteied In studying the configuration of test 
vcctois by the methods described heic, it is best not to include in the con¬ 
figuration any tests whose communalities aie small Such test vectons are 
slioit, and, when they are much extended, their projected location on the 
surface of the sphere is unstable If they are used m plotting the conligui a- 
tion, they should be specially marked so that less weight is given to them 
than to the test vectois which are not veiy much extended in normalizing 

+ — 



Figure 12 


The octants 

In Figure 12 we have lepiesented diagrammatically the right sphcncal 
triangle shown in the photogiaph of Figure 11 Tins diaguun will be used 
frequently to lepiesent the positive octant foi thiee-dimensional factoi 
problems, and it cun be used to advantage in discussing factorial principles 
extending to more than three dimensions The tlnce oithogonal axes /, II, 
III, are shown at the coiners of the co-mdinate triangle in Figure 12, 

A test vectoi that is colhnear with one of the co-oulmate axes would have 
zero loadings on all but one of the factors A test vector that is a lineai com- 
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bination of two reference vectors lies in the plane that is spanned by the two 
reference vectors. The test vectoi can then be defined by a vector equation, 
such as 

(19) iVjiXi + Wj 2 X 2 - J , 

in which w,i and w j2 aie the weights assigned to the test vector J for the 
two reference vectors Xi and X 2 If the two weights are equal, then the 
test vector J lies midway between the two leference vectors m the plane 
I-II of Figure 12. If the weight w,i is the larger, then the test vector J lies 
close to the first reference vector. If we write the equation of a test vector J 
for the three dimensions of Figure 12, we have 

(20) w,\Xi + w j 2 X 2 + w,aX 3 = J , 

111 which the weight w j3 is zero when the test vector J lies in the plane I-II 

The signs of the three weights determine the region in which the test 
vectoi J is located. If all three of the weights are positive, the test vector J is 
located inside the positive octant, as indicated by the signs H— 1 —f- in the 
positive octant of Figure 12 If the third weight is zero and the other two 
aie positive, we have the weights + + 0 , and the test vector then lies in the 
arc I-II of Figure 12. If the weights are 0+0, the vector J is collinear with 
the second reference vector, as shown m the figure. Similarly, if the weights 
aie +0 0, the vector J is collinear with the first leference vector. If the 

first weight is negative, we have thew eights-HO, and the vector J then 

lies in the plane spanned by the first and second reference vectors, as shown 
in the figure Similar relations hold for the weight combination + — 0 as 
shown 

Theie aie twenty-six sign combinations of the weights in equation (2) 
which define the region in which the test vectoi can be located 111 thiee di¬ 
mensions, the combination 0 0 0 being eliminated because it represents a 
null vector If one of the signs is zero, the test vector lies m one of the co¬ 
ordinate planes If two of the signs are zero, the test vector is at the inter¬ 
section of two of the co-ordinate planes The same reasoning can be extend¬ 
ed to n dimensions The fact that the weights of the equation are all posi¬ 
tive does not imply that the configuration can be inclosed within a right 
spherical triangle unless the reference vectors are separated by right angles 
or acute angles If the saturations of the tests in the several factors are all 
positive and if the factors are not negatively correlated, then the configura¬ 
tion can be inclosed m a right spherical triangle which becomes the positive 
octant This is a special case of considerable psychological interest because 
of the fact that mental abilities do not seem to show any negative correla¬ 
tions and psychological tests are positively eon elated, even when they rep¬ 
resent the most diverse tasks 



CHAPTER VIII 

THE CENTROID METHOD OF FACTORING 

In factoring a correlation matrix the pui pose is usually to account fm 
the coirelations with fewer faetoi s than theie are tests, so that r <n, where r 
is the numbei of factois and n is the number of tests With fallible data 
there will always be residual correlation coefficients aftei the extraction of r 
factors unless r = n, in which case the residuals can be made to vanish ex¬ 
actly. The reason is that the rank of a correlation matux for fallible data is 
always equal to its older. The factoring should be done so as to make the 
residuals as small as possible after each factor has been detemuned The 
least-squares solution foi this problem minimizes the le.si duals and maxi¬ 
mizes the factor loadings on each successive factor The resulting factorial 
axes are the pnncipal axes These would be deteimined loutinely in faetoi 
analysis, except for the computational labor that is involved foi laige test 
batteries of twenty or thuty or moie tests The eentioid method, like the 
grouping-method, is a computational compromise The mam eentioid axis 
for any given table of cori elation coefficients, or residual coefficients, can be 
regarded as a first approximation to the major principal axis The main een¬ 
tioid axis foi any given correlation matiix, 01 lesidual matux, is obtained by 
simple summational procedures aftei appropriate reflections of the test veo- 
tois with corresponding sign changes, and the results are often fauly close 
to the statistically ideal least-squai es axes In lai ge pi oblems 1 1 is sometimes 
desirable to deteimine an additional faetoi by the centroid method beyond 
what is required by the principal axes to obtain the same average magnitude 
of residuals The subsequent rotation of axes fiom either method of factor¬ 
ing usually shows one or more residual factors that do not contain enough 
variance for dependable interpretation 

Several theorems will be derived which involve the eentioid of the test 
vector termini These will be followed by numerical and geometrical exam¬ 
ples The theoretical examples m this chapter are exact, in that all conela- 
tional elements are assumed to be known and the rank r is less than n, so 
that the rth-factor residuals vanish identically An example will also be 
given showing computational methods that are suitable foi experimental 
data. 

Centroid theorems 

Consider, first, the fundamental equation of multiple-factoi analysis, 
namely, 

W Tjf, = OjiClu + Cliidkl + + UjrU/fer , 
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m winch the con elation between two tests, j and k, is expiessed in teinas of 
the factor loadings of these tests in r orthogonal factors Summing each 
column of the con elation matiix, we have 

(2) 2 Jj/c = a/.i s + + au 'y f g v 

) 3 * J 

Summing foi all columns, we have 

(3) SX’’* = 2 aiI X 0,1 + 2 CLk% Oj2 + + 

k 1 k l k 3 k J 


But 

(4) 


1 ft km — tijm y 


so that 

(5) 2 2 ,ji = [ 2 aji 


l j L o 


+ 


X® 1 ’ 

L 3 


+ + 


Let the sum of all the con elation coefficients (oi lesidual coefficients) be de¬ 
noted T t , then 


(6) XS rjt = ’‘' 

k i 

so that 

(V) r. = 2 

m 

We then have the inteiesting theorem that the sum of all the coefficients m a 
con elation matt ix is equal to the sum of the squares of the column sums in any 
con esponding oithogonal factor matrix 

Foi the puipose of illusti aling the theorems of this chapter, a small ficti¬ 
tious example was wnlten, as shown in Table 1, foi five vanables and two 
factois The inieicoirelations aie shown m Table 2 The sum of the cone- 
lations is + 29 Applying the theoiem to the sums of the factoi matrix, we 
have (— 2) 2 -f (+ 5) 2 = + 29, which is also the sum of the correlations In 
Figure 1 we have the eon esponding configuiation of five test vectors, which 
aie diawn without leference fiame 
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Table 1 

Fictitious Factor 
Matrix 



Table 2 

C 07 relation Mahix 




I 

II 


+ 1 

+ 2 

+ 3 

+ 4 

+ 5 

1 

- 9 

- 1 

+ 1 

82 

60 

- 33 

- 64 

- 58 

2 

- 6 

-.6 

+2 

60 

72 

- 06 

- 48 

- 00 

3 

4 

- 3 

+3 

- 33 

- 06 

25 

25 

12 

4 

7 

1 

+4 

- 64 

- 48 

25 

50 

46 

5 

.6 

4 

+5 

- 58 

- 60 

12 

46 

52 

V, 

- 2 

5 

2 

- 13 

18 

23 

09 

- 08 




|2| 

2 97 

2 46 

1 01 

2 33 

2 28 


29 



The centroid of a set of poults is the centei of giavity of equal weights at 
the points The centroid of the test vectoi tcimim has the eo-oidmates 



3 3 1 


If the axes are so placed that the centroid lies m the fust axis of reference, 
then the centroid has zero projections on all the lemainuig (? —1) co-oidi- 
nate axes Hence, 

(®) Q'll ~ O73 ~ — (lj T = (), 

i i i 

so that the r co-ordinates of the centroid aie 

°> °> >° 
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Substituting (8) in (7), we have 



where the first axis contains the centroid We then have the theorem that 
the sum of the coefficients in the correlation matrix is equal to the square of the 
sum of the first centroid factor loadings. 

The first co-ordinate of the centroid of the test vector termini is also its 
distance from the origin, since the remaining (r — 1) co-ordinates vanish 
Hence the distance of the eentioid from the origin is 

(10) d = - a,i 

i 

By (9) we have 

( 11 ) d-±VF t , 

in which the distance of the centroid from the origin is expressed in terms of the 
coefficients in the correlation matrix. 

In order to determine the loadings «,i on the centroid factor, substitute 
(8) in (2) Then we have 

(12) r ,k u*i J 

i 

and from (9) we get 

( 13 ) y r, k = gin Vr] . 

3 

For convenience, let the column sums of the correlation matrix be denoted 


(14) 

2 r,k = r k 

Then 


(}5) 

Tk = 0*1 v 7 , 

so that 


(16) 

Tfc 

a h i = — # 

Vr* 
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For computing, 


determine a multiplier m = ~r^- 


Then 


( 17 ) a-ki = mr k , 

by which the first-factor loadings are proportional to the conesponding 
column sums of the correlation matnx By (17) the first-factoi loadings arc 
numerically determined 

By equation (9) we have a useful check on the computations, namely, 
(9a) 2) 


Reflection of test vectors 

The correlation matrix of Table 2 represents a configuratron whrch is 
shown in Figui e 1 The fact that no reference frame has been drawn in this 
figure corresponds to the fact that no reference fiame is implied in the cor¬ 
relation matrix. A finst axis m the configuration of five vectois evidently 
should be in the general direction of the vectois 1, 2, 4, 5, since an axis in 
this direction through the configuration would make the projections the 
largest possible. It is of secondaiy consequence whether the positive sense 
of this first reference axis is in the direction of the test vectors 1,2, or in the 
direction of &■ 

Since the configuration is not immediately given by the con elation ma¬ 
trix and since it cannot be physically constructed or seen when the eoi re¬ 
lations involve more than three dimensions, it is a practical pioblem to de¬ 
termine the general direction of the longest dimension of the configuration 
by inspection of the correlation coefficients When that can be determined 
by inspection of the correlation matrix, several of the test vectors can be re¬ 
flected so as to move the centroid as far away fiom the origin as possible 
Then the first centroid axis can be located so as to account foi a large pait 
of the correlation coefficients, leaving their residuals as small as possible 

Several criteria can be used to ascertain which test vectors lie in or near 
the longest dimension of the configuration These criteria are not always 
consistent, since a configuration has extension in several diiections One 
criterion is to determine the absolute sum of the correlation coefficients, or 
residual coefficients, in each column That test vector which has the highest 
absolute sum is likely to be in or neai the longest dimension of the test con¬ 
figuration Another criterion is to find that column which has the hugest 
absolute sum, ignoring the diagonals One simple method is to select foi in¬ 
flection that test vector which has the largest number of negative coeffi¬ 
cients This process can be continued until each column has a majority of 
positive coefficients Another method is to select for reflection that test 
vector which has the highest communality Inspection of the correlation 
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matrix of Table 2 indicates tliat test vectois 1 and 2 should be leflected 
When these columns and rows have been leveised in sign, the lesultmg cor¬ 
relation matrix is shown in Tables, and the configuiation, aftei reflection, is 
shown in Figure 2 The centroid of the reflected configuration is shown at 
point c m Figure 2. 



Figure 2 


The computation of the fust-factor loadings is shown m Table 3 Sum the 
columns to find ri for each column Find the total sum r t and the multiplier 
to Apply the multiplier m to find the first-factor loadings a ,i as shown 
These loadings correspond to the variables with signs, as shown in Table 3, 
The same factor loadings are recorded in the factor matrix of Table £ for all 
variables with positive sign 

Table S Table 4 

Correlations after Sign Changes Factor Matrix 



-1 

-2 

+3 

+4 

H-5 



I 

II 

-1 

82 

60 

33 

64 

58 


1 

- 893 

- 149 

-2 

60 

72 

06 

48 

GO 


2 

- 740 

414 

+3 

33 

06 

25 

25 

12 


3 

304 

397 

+4 

64 

48 

25 

50 

46 


4 

701 

094 

+5 

58 

60 

12 

46 

52 


5 

686 

- 223 

Tk 

2 97 

2 46 

1 01 

2 33 

2 28 

r, = ll 05 

1 



CL]\ 

893 

740 

304 

701 

686 

Sa,i = 3 324 

i 

s 

058 

533 


m= 300828 



THE CENTROID METHOD OF FACTORING 


155 


The tlieoicm of equation (!)) t an be veuhert in Tabic 3 The .sum of the 
fiiat-factoi loadings, with variables 1 and 2 leveiscd, is 3 324, and (3 324) 2 = 
11 05, which is the sum of the coefficients m this table The fir.sL-faotoi 
loadings in Table 4 lepresent all vauables with positive sign as ougmally 
given 

The fiist-factor residual coefficients can be wntten by equation (1), in the 
form 


(18) 


i a ji Dr i 


i = 


l&ktT 


The fiist-factor lesiduals aie shown in Table 5 They were computed fiom 
the given coefficients in Table 2 and the fhst-factoi loadings, fiist column, 


Table 5 

First-Faclor Residuals 


1 

+ 1 

+ 2 

+ 3 

+ 4 

+ 0 

4-1 

023 

- 001 

- 059 

- 015 

032 

+2 

- 061 

172 

104 

038 

- 093 

+3 

- 059 

101 

158 

037 

- 089 

+4 

- 015 

038 

037 

009 

- 021 

+5 

032 

- 093 

- 089 

- 021 

019 

Ch 

- 079 

222 

212 

049 

- 120 

2 

- 080 

220 

21L 

048 

- 122 

1*1 

190 

528 

507 

120 

281 

Sr. 

- 103 

048 

053 

039 

- 171 

Neg 

3 

2 

2 

2 

3 


of Table 4 The check sums, Ch, of Table 5 were obtained fiom the column 
sums2of Tabled and the fiist column of Tabic 4 TJxample — 13— (—893) 
(+ 058) = — 079 The algebraic column sums aie shown m low 2 and the 
absolute column sums in low |2| of Table 5 The sums, omitting diagonals, 
aie shown in row 2r„, and the last low shows the number of negative co¬ 
efficients in each column. Oidinauly, all these lows aie not computed. 
They aie shown here to illustrate different methods of deteimining which 
vaiiables to leflect By the absolute sums we could lcflect test 2 By the 
sums, omitting the diagonal, we could stait by reflecting test 5; and, by 
counting the numbei of negative signs, we could lcflect eithei test 1 or 
test 5 

In Figure 3 we have the original configm ation of five points and the fiist 
and second centroid axes The correlation malax in Table G is of rank 1, 
and its configuiation is therefore of dimensionality 1. The one-dimensional 
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residual configuration is shown in Figuve 3, where the five test vectors are 
indicated by as many collincar arrows in the second centioid axis. These 



Ficiube 3 


residual vectors are seen to be the piojeetions of the five oiiginal test vectors 
on a one-dimensional subspace (axis II) orthogonal to the fiist axis When 
the residual vectois 1 and S are reflected, the resulting five vectors are uni - 
dnectional, and the second-factor loadings are then equal to the residual 
vector lengths. The computation of the second-factoi loadings is shown in 
Table 6, and they are lecoided with proper sign in the second column of Ta¬ 
ble 4- The second-factor residuals vanish, since the given correlation matiix 


Table 6 


Fvsl-Facloi Residuals after Sign Changes 



-1 

+ 2 

+ 3 

+i 

■— 5 

-1 

023 

061 

059 

015 

.032 

-f 2 

061 

172 

164 

038 

093 

+3 

059 

164 

158 

037 

089 

+4 

015 

038 

037 

009 

021 

—5 

032 

093 

089 

.021 

049 

Tk 

190 

.528 

507 

120 

284 

a ,2 

149 

414 

397 

094 

223 


r, = 1 629 
1 - 1 277 


m= 7835 
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of Table 2 is of rank 2 and the lank of each residual matnx is one less than 
the pievious correlation matrix 

The group centroid method 

The procedure to be described here is one of the simplest forms of the 
centroid method 

In Table 7 we have an eight-vai lable correlation matrix The last column 
shows the algebraic sum of each row, and it will be used for check purposes 
The check column is vacant in the given coi relation matrix. The first sum¬ 
mation row shows the absolute sum of each column The highest absolute 
sum is 4 06, so that test 8 is chosen as a pivot test In this column we choose 
several tests, in this case three, which have the highest correlations with 

Table 7 


First Factor Determined, by Group Centroid Method 


1 

1 

2 


3 

4 

5 

0 

7 

8 

Ch 

2 

1 

60 

27 


10 

- 53 

15 

03 

09 

- 24 


400 

2 , 

27 

94 

— 

38 

- 10 

48 

- 17 

- 51 

01 


500 

3 1 

19 

- 38 


57 

- 41 

- 54 

52 

00 

- 57 


- 500 

4 

- S3 

- 10 

— 

41 

05 

10 ■ 

- 20 

- 08 

47 


- 100 

5 

15 

48 

— 

54 

10 

80 ■ 

- 77 

29 

07 


1 240 

6 

03 ■ 

- 17 


52 

- 20 

- 77 

81 

- 47 

- 70 


-1 100 

7 

09 

- 51 


00 

- 08 

29 

- 47 

89 

43 


700 

8 

- 24 

03 

— 

57 

47 

07 

- 79 

43 

80 


800 

S|r,i | 

2 00 

2 88 


3 24 

2 00 

3 30 

3 82 

2 82 

4 00 



3 

0 

0 


0 

0 

4-1 

-1 

0 




Sr.l 

- 12 

08 

— 

1 03 

83 

2 30 

-2 37 

1 19 

2 32 

3 20 

3 20 

U 

0 

0 


— 1 

4-1 

4-1 

-1 

+i 





- 75 

46 

~ 

2 65 

1 81 

3 23 -3 02 

1 04 

3 79 

4 30 

4 30 

O;! 

- 182 

109 


020 

440 

785 - 880 

471 

921 

1 045 

1 044 




T 

- 16 04 

v? 

« 4115823 

m =» 

242905 





2a u i - 4 117 (4 117)* ~ 10 OS 

test 8, disregarding sign These conelations aie r 68 = + .67, ro 8 = — 79, and 
rs8 = + 86 In practice these subgroups may contain three to five tests or 
more, provided that only those tests are included in the subgioup which 
have significant correlations with the pivot test 

The three tests in the subgroup are 5, 6, and 8, and the signs of their cor¬ 
relations with test 8 are +, —, and + , respectively In row s we record unit 
weight with these signs for columns 5, 6, and 8, and zeios foi the rest, us 
shown 

In the next row we record the column sums for these three rows with signs 
The first entry m row 2 r sk is obtained from the first column, namely, 
(+,15) —(+ 03)-t-(— 24) = — 12. The other entries aie obtained m a sim¬ 
ilar manner The check sum is obtained from the 2 column, namely, 
1 24=0) ( 1 100)+ ( + 860) = +3 200, which is the expected sum 

The actual sum of the row is the same, and it is recoided in the 2 column 
Note, next, the highest absolute sum m this row, which is -2 37 As a 
practical, but somewhat arbitrary, rule, we note that one-third of this sum, 



Table S 

Second Factor Dele) mined by Group Centtord Mcthod 



1 

2 

i 

1 

5 

h 

7 

8 

Ch 

2 , 

1 

2 

467 

290 

077 

- 450 

20} 

— no 

17b 

- 072 

650 

651 

J90 

928 

- 312 

- 148 

301 

- 074 

- 561 

- 070 

416 

447 

3 

4 

077 

- 312 

186 

- 137 

— 053 

- 026 

352 

001 

087 

088 

— 450 

- 148 

- 137 

456 

- 245 

127 

- 287 

065 

— 619 

- 619 


293 

m 

- 051 

- 245 

244 

~ 070 

- 0 80 

- 05 3 

420 

421 

o 

— 130 

~ 074 

- 026 

127 

- 079 

0 36 

— 056 

020 

- 181 

- 182 

7 

176 

- 561 

352 

- 287 

~ 080 

- 056 

60S 

— 004 

20S 

2U8 

8 

- 072 

- 070 

001 

065 

- 053 

020 

~ 004 

012 

- 102 

- 101 


1 95> 

2 777 

1 144 

1 91} 

1 441 

518 

2 181 

207 



a 

2,r*fc 

0 

330 

+1 

2 10 r > 

-*1 

- 903 

0 

031 

+ 1 

771 

0 

- 071 

-1 

-1 601 

0 

- 120 

572 

572 

u 

2r„* 
a ” 

+1 

797 

+1 

2 485 

-1 

— 826 

0 

- 419 

-H 

1 064 

0 

- 201 

— 1 

-1 185 

0 

- 102 

1 222 

1 223 

309 

963 

- 320 

- 162 

412 

- 078 

- 576 

- 074 

473 

474 


T - a 857 Vy = 2 580116 m = 387579 


2n„i = 2 580 (2 580)' = 0 656 


Table 0 

Third Factor Determined, by Group Centroid Method 



1 

2 


3 

4 


5 

fl 

7 

8 


s 

1 

372 

- 008 


176 - 

mm 


166 - 


354 


505 

.505 

2 

- 008 

001 

— 


008 

— 

003 

001 

- 00b 


- 000 

- Oil) 

3 

176 

- 004 


084 - 

180 


M3X 1 

051 

168 

- 02.5 

2io 

240 

4 

- 400 

008 

— 

189 

430 

— 

178 

114 

- 380 

053 

- 512 

— 542 

5 


- 00.1 


079 - 

178 


074 - 

047 

157 

- 023 

220 

225 

6 

- 106 

001 

— 

051 

114 

— 

047 

030 

- 101 

014 

- 145 

- 146 

7 

354 



108 - 

380 


157 - 

101 

35b 

- 047 

481 

481 

8 

- 019 

ms! 

- 

023 

0)3 


023 

014 

- 047 

007 

- 0i16 

- 067 

Sloil 

1 631 

032 


774 

1 752 


727 

101 

1 510 

217 




“I 

0 


0 

+ 1 


0 

0 

-1 

0 




-1 120 

022 

— 

533 

1 210 

—• 

501 

221 

-1 070 

149 

-1 52S 

-1 528 


-1 

0 


-1 

1-1 


-1 

+ 1 

-1 

0 




-1 571 

030 

— 

747 

1 691 

— 

701 

449 

-1 106 

209 

-2 130 

-2 141) 

0*3 

- 610 

012 

— 

289 

65) 

— 

272 

171 

- 580 

081 

- S29 

- 82‘) 




T 

= 6 658 

Vt 

— 

2 580310 

m 

= 3S7530 





So OJ = 2 580 (2 580) 2 = 6 Bob 


Table 10 


Factor Matrix by Group Centroid Method 



r 

II 

III 

A* 

1 

- 182 

309 

- 610 

501 

2 

109 

963 

012 

939 

3 

- 020 

- 320 

- 289 

570 

4 

440 

- 102 

655 

649 

5 

785 

412 

- 272 

860 

6 

- 880 

- 078 

174 

811 

7 

471 

- 576 

- 580 

890 

8 

921 

- 074 

081 

860 
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which is about .80, and all columns whose sun us are loss than thus number 
are given a weight of zero m the next row, u, as shown The other columns 
are given weights of +1 oi — 1 amnding to the signs of Xr,* as shown 
With the weights u we find the column sums Si,,*., which aic lecorded in 
the next row. The first entry in this iow is obtained fiom the first column 
with the weights given in the iow u It is — (+ 19) + (— 53) + (+ 15) — 
03) + (+.O9) + (— 24) = — 75 The check sum, +4 30, is obtained with 
similar weights fiom the X column, and it agrees with the actual sum of the 
entries m this iow 

Summing the entnes in the iow and applying again the same weights, we 
have 

+ 2 55 + 1 81 + 3 23 + 3 62 + 1 94 + 3 79 = 16 94 = T , 


fiom which we obtain the multiplier in — 



= 2429G5 


Applying this 


multiplier to iow Xr„* and to its sum 4 30, we have the factoi loadings a,i 
witli propei signs and then expected sum, 1 045 The actual sum of the iow 
is 1044, and it is recoided m the X column Finally, the algcbiaic sum of 
the row with weights u is 2 a ul = 1 117, and the square of this sum is equal 
to the value of T oi 1 1 of equation (9a) The factoi loadings in iow a,\ are 
copied duectly in the fust column of the factoi matii\ in Table 10 

Table 8 shows similar computations for tlio socond-factoi loadings The 
check column is obtained fiom the 2 column of Table 7, the fiibt-factoi 
loadings, and their sum, 1 045 The check sum in the first iow of Table 8 
is obtained thus' +.460— (— 182)(1 045) = + 050 The actual sum of the 
lesiduals in the first iow of Table 8 is + 651, as leeoidod The socond-factoi 
loadings, a, 2 , aie recorded in the second column of Table 10 

Table 9 shows the corresponding computations for the tluid-factoi load¬ 
ings The thnd-factor lesiduals computed fiom Tables 7 and 10 vanish in 
the range + 001 

The computational loutine for what we have called the “gioup centroid 
method" is so simple that a clerk can easily leain it Attention .should bo 
called to the fact that the subgioup s should include thiee 01 moio tests 
If only one pivot test weie used and if the weights u were made +1 01 — 1, 
omitting the zeio weight, then this method would induce to what was 
called the "simplified centroid method," which was published m 1033. + 
That method was defective, in that the eominiuialities of the pivot tests wei e 
too high, sometimes even exceeding unity Mm cover, the method did not 
properly separate the common factors fiom the unique factons The method 
which has been described here and which has been called the "group cen¬ 
troid method” avoids these difficulties by using the subgroups s and by using 
the thiee weights for u However, the gioup centroid method is probably 

*A Simplified Multiple Factor Method (University of Chicago Booksluie, 1033) 
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not so effective as the complete centroid method in separating the common- 
factor space from the unique test space 

The complete centroid method 

The complete centroid method which has been in use in the Psychometuc 
Laboratory for a number of years will be described here. This description 
trill be given in terms of the computations on a standardized data sheet, 
which is illustrated in Tables 11-13 inclusive The complete centroid 
method will be described with a numerical example of twelve vaiiables, and 
we shall include here a description of all the numerical checks which consti¬ 
tute a part of our computational routine. The computational checks aie 
explicitly shown on the data sheets 

By the complete centroid method we mean the successive reflection of 
variables, one at a time, until all the column sums in the correlation matrix, 
or the lesidual matrix, are positive or zero These column sums include the 
diagonal entries No matter what the signs may be in a given correlation 
matrix, one can always reflect some of the vaiiables so that all the column 
sums become positive 01 zero In the complete centroid method this work 
is done systematically by reflecting one variable at a time While it is al¬ 
ways possible to make all the column sums positive or zeio by reflecting 
some of the variables, the factoiial solution is, in general, not unique If 
simple structure exists in the test configuration, the rotated factor matrix 
is unique. It is independent of the factor matrix that is obtained by the 
centroid method The lack of uniqueness of the centroid method of factor¬ 
ing constitutes, then, no problem because the rotated factor matrix is unique 
and the structure is unambiguous 

The following paragraphs are numbered to show the successive computa¬ 
tional steps in the complete centroid method with successive adjustment 
of the communalities ■ 

1 A given correlation matrix with twelve variables is recorded in Ta¬ 
ble 11 The diagonal entries are left blank, as shown This example 
is intended to show the computational routine that is used on expen- 
mental data for the particular case in which it is decided to adjust 
the diagonals after each factor has been extracted 

2 The column sums are recorded m row s, and the row sums are re- 
coided in the column 2 The sum of all the entries in the correlation 
matrix is 79 40, which is recorded at the intersection of row s and 
column 2 

3, Since all the given correlation coefficients are positive in this exam¬ 
ple, we omit several steps that aie required for subsequent factors. 
Row s is copied into row C , as shown. 

4 The communalities are recorded m row d Here the computer decides 
how to treat the unknown communalities. He may use one of the 
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estimation formulae foi the communalities and record the estimat 
m this row Several such formulae aic described m chapter am 
he decides to use unit communalities, these may be recoided 
iow d The simplest estimation foimula is to take the highest c 
efficient in each column as the estimated communality, and this 
the method used in the piesont numerical example The data she 
is designed so that it is applicable for any method of estimating 
computing the diagonal values Since there have been no si 
changes m this table, row d is copied into iow D, as shown in Tai 
11 

5 Row E gives the sum of C T D 

6 The entry 88 32 in iow E is the sum of the cntucs C and D imme< 
ately above it Row E is then summed If the sum agices with t 
alieady iccoidcd sum, this fact is indicated by a check maik, 
shown 

7 The fust entry m the small table at the lower light cornel is 88 ! 
which is obtained fiorn iow E It is denoted T 

8 The value is next lecoided, as shown 

9 The iccipiocal m is recoided m the next space 

10 The product mT ns equivalent to 9 40 and is lecoided in the m 

space _ 

11 Next, it is voiihed that this value checks with V T This vcuficati 
is indicated by the check mailc, as sliown 

12 The last row of the table shows the factor loadings, which aie t 
pioducts mE, as shown Applying the same multiplier to the che 
sum gives m27? = 9 40, as shown 

13 The algebiaie sum of (lie last iow is 9 39, as recoided in the 2 o 
umn The fact that llic.se two sums agiee, namely, 9 39 and 9 40 
indicated by a cheek marie 

14 The sum of the absolute values of the entiles in the last iow (9 39 
iccoidcd in the small table The fact that it agiees with the va 
of mT is shown by a cheek mark 

15 The filst-faetor loadings are recorded in the first column of the fac 
matnx F in Table 15 

The calculation of icsiduals 

The compulations fin the first-faetoi icsiduals and for the second-fat 
loadings nic eai i led out on a new data sheet, as shown in Table 12 The s 
ccssivo steps m these computations aie as follows 

1 The hrst-faetm loadings aie recoided in column +a, as shown 9 
sum of tills column (9 39) is also recoided 

2 In the top iow (5= —a) the same faetoi loadings are lecorded w 
leversed sign The sum of the top iow is — v= — 9 39, as lccoid 
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3 The first-factor residuals (p,0 are ordinarily expressed in the form 

Pjk — r Jk — djidki 

Since the fust-factor loadings, a,i and a ku may be positive or nega¬ 
tive and since their product is preceded by a minus sign, the com¬ 
puter is ordinarily required to keep m mind these three signs The 
computational routine is simplified by expressing the residual in the 
form 

Pjk = I’ll. + (otj 1 ) (— a ll) 

The factor loading (a jl ) is represented in the column (+a) and the 
factor loading ( — am) is represented by the top row ( b ) on the data 
sheet The actual computation of residuals will be illustrated for the 
first correlation coefficient, namely, r 2 i — + 17 in Table 11 The resid¬ 
ual pn is to be recorded m the cell entry (21) in Table IS The factoi 
loadings 70 and — 59 are found m column a and row b directly op¬ 
posite the cell entry (21) The actual computation for this residual 
then is 

17 + (-70) (- 59) = - 24 

The residuals in the lest of the column for the first variable aie com¬ 
puted m the same manner The diagonal cells are left blank in the 
residual table. 

4 The check sum (— 25) for this column of residuals is next recorded 
The value s is obtained fiom the pievious data sheet In detail the 
computation for this entry is 

4 94 + (9 39) (- 59) + (- 59) 2 = - 25 

5 The sum (— 23) of the first column of residuals is recorded in row s 

6 If this sum s agrees closely with the check sum Ch, as m the present 
case, it is assumed that the computations were done correctly The 
slight discrepancy is to be expected from rounding. When the agree¬ 
ment of the sum and the check sum has been noted, the first column 
of residuals is copied into the first row of the table The sum (s = 
— 23) is copied into column 2 

7, The transcription in the first row is verified by actually summing that 
row The agreement of the actual sum with the recorded sum is in¬ 
dicated by the check mark. 

8 The residuals for the second variable aie next computed, beginning 
in cell (32) When the residuals m the second column have been com¬ 
puted, they are verified m the rows Ch and s as before The second 
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column of residuals is then copied into the second row and verified by 
adding the second row The rest of the residuals are computed in the 
same mannei It should be noted that the number of residuals to be 
computed is leduced m the successive columns When the computer 
reaches column 11, he has only one lesidual to compute When the 
computer reaches column 12, there are no new residuals to compute 
The last column is verified by the check sums 
9 The final check for the whole table of residuals is obtained m the 
form 

2s - z> 2 + 25 2 

Its numerical value (—1 32) is lecoided as shown 

10 The sum ( — 1 30) of the low Ch is next leeorded The fact that this 
sum ( — 1 30) agrees closely with the pi eviously recorded sum ( — 1 32) 
is repiesented by a check mark m the small square * This verifies the 
computations in the row Ch 

11 The sum ( — 1 28) of the row s is next recorded The fact that this 
sum agrees closely with the two sums just leeorded is represented by 
a check mark, as shown in the second small square 

12 The entiies in column 2 aie summed The fact that this sum agrees 
with the alieady recorded sum (—1.28) is lepresented by a check 
mark adjacent to this sum 

he sign changes 

1 If all the entries m row s are positive, then no i eflections ai e necessaiy. 
This was the case in Table 11. However, in Table 12 row s contains 
negative entiles Hence we proceed by lecording the entiies in row A 
The entiies m low A are leeorded to three decimals The pievious cal¬ 
culations have been recorded to two decimals The anticipated sum 
( 640) is recorded, as shown Row A is next summed The fact that 
the sum agrees with the expected sum is represented by a check mark. 

2 The highest positive number in row A is 120 This number is written 
in parentheses, and it occurs in the second column This fact is rep¬ 
resented by writing + (2) at the beginning of the next row. 

3 Each entry in this row is the sum of the corresponding cell entry of 
row A and the second row of residuals For example, the first entry is 

( 115) + (-.24) = - 125. 

In the sign-change calculations the diagonal entries are treated as 
zeios Thus the second entry in row +(2) is the same as the second 


The boxes in the printed data sheet are represented here as parentheses 
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entry m row A Each entiy m this 1 ow is wntten m pai enthe&es if the 
corresponding entiy m row A is m paienthcses 

4 The anticipated sum. (400) is obtained from the coiresponding en¬ 
tiles m column S, namely, 640 and - 24 The sum of the new low 
agrees with the anticipated sum, 400 This fact is lepresented by the 
check mailt 

6 The highest positive sum in this row is 275, and hence this number is 
wntten m parentheses This number is m column 9, and therefoie the 
next low is denoted + (9) as shown Each entiy in row +(9) is the 
sum of the corresponding entries in low +(2) and in the ninth low of 
lesiduals For example, the fust entiy is 

(- 125) + (- 19) = - 315 

The sums m low +(9) are wntten in paientheses if the corresponding 
entiles in the pievious rows aie m parentheses It will be seen that 
the entries in these columns aie wntten in parentheses after the sign 
change for that column The entnes in row +(9) are summed and 
checked as before This procedure continues until all the entiles in a 
low aie negative, zeio, oi positive in paientheses 

It happens occasionally, especially with a large test batteiy, that a 
test vector is leflected twice In determining the next vanable which 
is to be reversed, we choose eithei the largest positive number without 
parentheses or the laigest negative number in parentheses, whichever 
has the laiget absolute value The same result is obtained if we regaid 
the paientheses as the equivalent of a negative sign and if the mle is 
adopted of reversing that vanable which has the laigest positive value 
If, accoidmg to the rule, a variable must be leveised which has pievi- 
ously been leversed, this can be accomplished by subtiactmg the cor- 
lespondmg low of residuals instead of adding, as in the regulai pio- 
cedure 

6. When all the entiles inarow are zeio or negative, counting paienthcses 
as the equivalent of a negative sign, this low is denoted B, as shown m 
Table 12 

7 The entnes m low C are lecoided to two decimals 

8 The weighted sum of the twelve entiles in column 2 is recoided in 
iow C Each weight is either +1 or — 1 The weight is +1 if the vai i- 
able has not been reveised, and it is —I if the variable has been re- 
veised The weighted sum of the twelve entnes in column 2 is then 
— 12, as shown 

9 The entnes in iow C aie summed The sum must equal the antici¬ 
pated sum, which here is — 12 This agreement is repiesented by the 
check mark 
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The communalities 

1 In low d aie leeoided the estimated diagonals Each entry in low d 
is the laigest absolute value of the residuals For example, m the 
second column the residual with largest absolute value is — 24 It is 
leeoided in row d with positive sign. 

2 In row D the same values are recorded with the signs indicated in 
row C The weight, g, is +1 for unrcflected test vectors and — 1 foi 
reflected vectors The sum (18) of row D is recorded as shown 

Calculation of factor loadings 

1 Row E is the sum of entnes in rows C and D The anticipated sum 
(06) is, similarly, the sum of 2-entries of rows C and D Row E is 
summed The fact that the sum of i ow E is equal to the anticipated 
sum ( 06) is indicated by the check maik 

2 The sum of the absolute values m row E is 10 60, which is iceoided m 
the small table _It will be noted that 2|F| = 'Ly.E 

3 The value of ^T is next lecorded in the small table 

4 The reciprocal m is recorded in the next space m the small table 

5 The product mT is leeoided m the next space of the small table It is 
recorded to two decimals Its agieement with Vt is indicated by the 
check mark 

6 The factor loadings aie iccorded m the last low of Table 12 The an¬ 
ticipated sum, ml,E= 02, is lecorded in the check column 

7 The sum of the factor loadings ( 03) is leeoided m the 2 column The 
fact that these two sums agree closely, namely, 03 and 02, is indi¬ 
cated by a chock mark in the small square 

8 The sum of the absolute values (3 27) of the faetoi loadings is recorded 
in the last space of the small table The fact that this sum agrees 
closely with mT is indicated by a check maik in the small squaie 

The same routine is followed in computing the second-factor residuals, 
which are shown m Table IS The third-factoi lesiduals are shown m Table 
14 These residuals aie too small to justify further factoring 

The factor loadings aie assembled m the factor matux F of Table 15 The 
fouith column of this matrix shows the communalities as determined by the 
centioid method, with successive adjustment in the diagonal cells The last 
column of this table shows the tiue communalities, which are consistent with 
the side conelations of the given couelation matnx, which is of lank 3, 

When the tiue communalities aie known, as in tho present fictitious ex¬ 
ample, the same data sheets can be used for the centioid method in such a 
way that the third-factor residuals vanish identically The obtained com- 
munalities aie then exactly equal to the fine communalities The procedure 
is then to lecoid the tiue communalities 111 low d of Table 11 In computing 
the lesiduals foi Table 12 the communalities are tieated m the same way as 
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the coi relation coefficients The lcsidual communalities aie then lecoidecl 
in row d of Table 12 without any adjustment When this pioceduie is fol¬ 
lowed, the third-factoi residuals vanish identically, including the eommu- 
nality residuals, and the obtained communalities are equal to the ti ue com¬ 
munalities This procedure would be used with experimental data except 
for the fact that the true communalities are unknown and hence must be 
estimated In dealing with expeiimental data the residual communalities 
are adjusted after each factor has been extiacted in the manner which has 
been described The computei may piefei some othei method of adjusting 
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the estimate of the communalities for each lesidual table by using one of 
the estimation formulae m chapter xm 

A multiple group method of factoring the correlation matrix 

Most of the methods of factoring the correlation matnx requn e the cal¬ 
culation of residuals aftei each factoi has been extracted This is perhaps 
the most laborious part of factoring The method to be descubed here 
avoids the computation of residuals after each factoi has been computed 
Since the method turns on the selection of groups of test vectors, it will be 
called a multiple group method of factoring * The method can be used for ex¬ 
tracting one factor at a time if that is desned, but it will be considered heie 
for the more interesting case in which seveial groups of tests aie selected 
from the correlation matrix at the star t The result of this method of fac¬ 
toring is a factoi matrix F which satisfies the fundamental i elation FF' ~ R, 

The multiple group method of factoung is general, m that it can be used 

* L L Thui stone, “A Multiple Group Method of Factoung the Con elation Matnx,” 
Psychometrtka , X (June, 1945), 73—78 
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on a con elation matrix of any rank, any oidei, and any configuration of test 
vectoiB, and the method is successful foi any aibitiaiy grouping of the 
tests, piovided that the seveial gioups of tests aie lmeaily independent * 
This method of factoung has a limitation, in that the coininunalities aie as¬ 
sumed to be known at the start This is not a senous hardship, because the 
factoring can be repeated with the communalities that aie found in the first 
trial and the convergence toward the conect communalities can be expected 
to be rapid. 

This method of factoung is based on the well-known principle that a 
common-factor space of r dimensions is spanned by any set of r linearly in¬ 
dependent vectors m that space The group centroid vectois are used for 
this purpose 

The method of selecting the gioups of tests is not crucial for the multiple 
gioup method of factoung, since the only requirement is that the centroid 
vectors for the seveial gioups shall be hneaily independent This condition 
is neaily always satisfied without taking any special pieeautions about it. 
With expei imental data it would be a laie case m which this condition 
would not be satisfied. If, by chance, the centroid vectors for the several 
groups should happen to be linearly dependent, then this fact would become 
appaient m a latei step of the calculations in which an inveise is computed 

In selecting the seveial groups of tests for the purpose of factoring, the 
compute! will natuially put together the tests that have high mtercoirela- 
tions He will select, foi each group, those tests which are neaily colhneai, 
if such groups can be found in the coi relation matrix If such groups are not 
readily seen by inspection, then the grouping can be carried out by some 
othei routine One such routine is as follows Select, as the first pivot, a 
test which has one or moie significant correlations and which also has an 
appieciable numbei of low coefficients, if these can be found Anange as a 
group those tests that have significant correlations with the first pivot test, 
either positive or negative Retain, as the first group, those tests which, by 
leflection if necessaiy, produce a subgioup in which all correlations are posi¬ 
tive or zeio. Choose the next pivot test from the lemaimng tests in the 
same manner Proceed m this way until all the tests with significant coi re¬ 
lations aie represented m the several gioups The grouping of the tests can 

* Holainger has described a factoring method which anticipated the first two parts of 
the multiple group method of factoung, namely, the computation of an oblique factoi 
matux V by summations and the computation of the correlations between the group cen- 
tioid axes ("A Simple Method of Factor Analysis,” Psychometnka, Vol IX [December, 
1944]) The multiple gioup method adds a thud step, namely, the computation of an 
oithogonal factor matrix F, which is the staitmg-pomt foi the lotational pioblem In de- 
sciibmg his factoring method, Holzinger limits the method to correlation matrices which 
can be divided into sections of unit rank However, Ins method is goneial, and it can be 
used on coirelation matnees of any order, rank, and configuiation His method is alge- 
biaically equivalent to the fiist two steps in the multiple gioup method 
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be carried out in any one of seveial ways for the multiple group method of 
factoring. 

In Table 16 we have a numerical example which illustrates the multiple 
gioup method The given eonelation matrix has eight variables with mtei- 
coirelations as shown The commiinalities were known in this case With 
experimental data the commiinalities must be estimated In the present ex¬ 
ample, the grouping of the tests was quite arbitiary, with little refeience 
to the correlations Tests 1, 2, 6, and 7 were placed in group 1, and tests 
3, 4, 5, and 8 were placed m gioup 2 Only two gioups weie used m this 
example The natural proeeduie is to group the tests into correlated clus¬ 
ters as far as possible The multiple gioup method is not dependent on the 
grouping, and hence the groups were selected arbitrarily for this numerical 
example. 

In row Si we have the sums of columns for rows 1, 2, 6, and 7 In low Si 
we have the corresponding sums for rows 3, 4, 5, and 8 The eonelation 
matrix has now been augmented by two lows, but the rank is unchanged 
because these two rows are linear combinations of the given eight rows If 
two similar columns were added to the right of the eonelation matnx, we 
should have an augmented correlation matrix of the same rank This has 
not been done in Table 16 because it would simply be the transpose of the 
two rows of matrix S, which are already computed and recorded in the table 
The sum of the conespondmg entries in each row of the matnx (S' are 
then computed and recorded, as shown in matnx T. For example, the entiy 
1 28 is the sum of columns 1, 2, 6, and 7, of the fiist row of S The othei 
sums are computed in the same manner The square matrix T is of order 
2X2, and it represents the lower right corner of the augmented correlation 
matrix in which the rank is unaltered 

The next step is to compute the reciprocals of the square roots of the di¬ 
agonal elements of the matrix T. These are the weights Wi and io 2 , which 
aie shown to the right of the matrix T 

The matrix V' is computed by applying the weight ioi to the first row 
of S, and the weight w 2 to the second row of S Similarly, the same weights 
on the rows of T give the square matrix U, as shown The matrices V' and 
U may be thought of as two rows by which the correlation matnx could 
be augmented instead of the lows of S and T The matrix U would then be 
the lower right corner of the augmented eonelation matrix, in which the 
lank is still unaltered 

The two weights, wi and w 2 , are applied to the columns of the ma¬ 
trix 17, and we then have the matrix R vq with unit diagonals The calcula¬ 
tions are in part checked by the unit diagonals of this matnx. This matrix 
shows the cosine of the angular sepaiation of the two unit leferenco vectois 
that have been chosen Each one of them contains the centroid of a group 
of test vectors 
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If we write the tianspose of the matrix V, we have an. oblique factor ma¬ 
trix V, as in previous methods of rotation from the orthogonal factor ma¬ 
trix F, but we have obtained here the oblique matnx directly and without 
computing the orthogonal factor matnx F, which is usually obtained first 
by the various factoring methods 

Since we have here an oblique factor matrix V, as well as the cosines of 
the angular separations of the unit lefeience vectors for the factoi matrix V, 
we can deteimine the ti ansformation which should carry the oblique factor 
matrix V into an oithogonal factor matnx F Since the location of an or¬ 
thogonal frame m the system defined by V and U is not unique, wc must 
first define in some convenient way the location of the desued orthogonal 
reference axes of F We shall do this by locating the first of the orthogonal 
refeiences axes collinear with the first of the oblique axes Then the fust 
column of F will be the same as the first column of V. The next 01 thogonal 
reference axis will be located in the plane of the first two oblique axes and 
orthogonal to the fust one. This process is, in fact, the diagonal method of 
factoring applied to the correlation matrix R Pq When this is carried out 
routinely, we have the factor matnx F pm) in which the two oblique ref¬ 
erence axes p aie defined m the orthogonal frame m 

The inverse of the transpose of this matnx gives A^£ = (F',„) _1 We can 
now wnte the transformation by which the orthogonal factoi matrix F can 
be written for the whole test batteiy The matrix F pm then is the additional 
two rows of F which describe the oblique reference axes p This transfor¬ 
mation is 

(19) FA -1 = F , 
from which we have 

(20) FA = V , 

where A is already numerically given. 

Applying this transformation to the matnx V by equation (19), we ob¬ 
tain the factor matnx F, which is shown in the upper right coiner of 
Table 16, 

When this factor matrix has been written, a residual table can be com¬ 
puted foi the two factors, to determine whethei additional factors need to 
be extracted In the present example, which is exact, the lesiduals vanish 
identically. In a piactical problem with experimental data, the residual 
table should be computed and inspected to deteimine whether additional 
factors should be extracted If that is desirable, one proceeds with the 
lesidual table as with the original correlation matrix m the present example 
Groups are then listed from the residual correlation matrix until the tests 
with significant residuals seem to be represented The process is repeated 
so as to add one or more columns to the factor matrix F 
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In the present case we have one set of residuals to compute for the entire 
factor matrix F instead of several sets of residuals, as would have been re¬ 
quired in most of the current methods of factonng the con elation matrix 
It will also be seen that the older of the inverse to be computed is equal to 
that of the number of factors determined by this process For example, if 
sevenfactors are to be deteimined and if the piocess starts with seven lineal¬ 
ly independent clusters, then there is an inverse of order 7X7 to be com¬ 
puted. If the factoring pioceeds with, say, only foui groups, then there is 
an inverse of order 4X4 to be computed, and the fouith factor residuals 
should then reveal three additional groups, resulting m three columns to be 
added to the factor matrix F The computational labor is probably mini¬ 
mized by finding as many groups in the given eon elation matnx as there 
are common-factor dimensions, so that only one table of coirelational 
residuals needs to be computed However, if the first estimate of the num¬ 
ber of clusters is too small, one merely repeats the process until the residuals 
vanish. If the factoring is done in several steps, the inverses are, of course, 
of smaller order 

The grouping was chosen quite arbitiarily in this example in ordei to ll- 
lustiate in a numerical example that the multiple group method of factoring 
is independent of the method of grouping the variables The only require¬ 
ment is that the centioid vectors of the several gioups must be hneaily in¬ 
dependent 

It is not necessary to select the same number of tests in each group, but 
there should be at least three or four tests in each group This lequirement 
is associated with the separation of the common factors from the unique 
factors A limiting case is that m which each group contains only one test 
The multiple group method then degenerates into the diagonal method of 
factoring,* The multiple group method is not quite foolproof A computer 
might assemble a group of tests whose sum would give a null vector. It 
would then be necessary merely to alter the grouping 

* L L Thurstone, Theory of Multiple Factors (Ann Aibor, Mich Edwards Bros, 
January, 1933), pp, 13-16 



CHAPTER IX 

CONFIGURATIONS AND FACTOR PATTERNS 
Interpretation of factorial structure 

We have seen that every con elation matnx implies a configuration of test 
vectors and that the number of dimensions of the configuration is the rank 
of the correlation matrix It has also been shown that a reference frame 
must be inseited in the configuration before a factor matrix can be written 
The combination of the configuration and the reference frame is called a 
“structuie ” The problem to be considered now is how to locate the ref¬ 
erence axes in relation to the configuration so as to give the most plausible 
interpretation foi the reference frame All students of the multiple-factor 
problem agree that the refeience axes should be placed m some relation to 
the configuration, but they differ on the criteria by which this should be ac¬ 
complished These differences, which are the subject of considerable con¬ 
troversy, aie due principally to the different purposes of factor analysis 
which the sevei al writers have had in mind What is correct and acceptable 
for one purpose may be n relevant for another purpose 

At the outset of a factor analysis, the investigator must decide whether 
he will write a factoi matrix to account for the complete variance of each 
test or only the common-factor variance His choice will depend on his 
puipose If he decides to account faetorially foi the complete variance of 
each test, then he wntes unity in the diagonal cells of the correlation matrix 
The several methods of factoring are applicable in this case, just as well as 
m the case where communalities are inserted m the diagonal cells. The fac¬ 
tor matrix will then repiesent a set of n unit test vectors, which define the 
total test space This is a subspace of the total factor space, which involves, 
in general, (n+r) factors 01 dimensions The configuration will not involve 
any more dimensions or factors than there are tests in the battery, and it 
will reproduce the mtercoi relations exactly, as well as the diagonal unit 
self-correlations If the object is merely to write a factor matrix which re¬ 
produces the correlations, then the task is easily done by any one of several 
factoring methods The reservation about this solution is that it is difficult 
to identify the factors, common and unique, from such a factor matrix; but 
that may not be the purpose of the investigator who proceeds in this way 
If the purpose of a factorial analysis is to identify the functions or param¬ 
eters that produce the intercorrelations, then the pioblem can sometimes 
be solved by limiting the analysis to the common factors which are detei- 
mined by the reduced correlation matrix If this is the purpose of the anal¬ 
ysis, the communalities are inserted in the diagonal cells of the correlation 

176 
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matrix, If theie is reason to suspect that there is no unique variance m the 
measurements, then unity can be used in the diagonal, but such situations 
aie raie m those fields m which factor analysis is likely to be most useful 
With the communalities m the diagonal, we have the possibility that the 
number of common factois is considerably less than the numbei of measure¬ 
ments, so that the pioblem becomes soluble, at least as far as the numbei 
of unknown common factors is concerned. 

The purpose of a factor analysis also determines how the data are to be 
treated, even after a decision has been made concerning the self-correla¬ 
tions The two conflicting purposes that are largely responsible for contro¬ 
versy in this field are (1) the statistical condensation of the correlations into 
a factor matrix with maximum effectiveness and (2) the intei pi etation of 
the factors which lepiesent distinguishable functions or parameters. Ever 
since multiple-factor analysis was introduced, it has been known that these 
variant solutions are mathematically equivalent The debate is concerned 
with the choice of reference frame for the factor matrix, which is the end- 
product of the computational work and the starting-poml foi intei pretation. 
These two purposes may oveilap when an investigator tries to accomplish 
both the maximum statistical condensation of the correlation coefficients 
into a factoi matnx and also a physical intei pretation of the factors as rep¬ 
resenting fundamental concepts or functions. Some students of factor anal¬ 
ysis deny that meaningful factors exist in psychological measurements oi 
performances, at least when they are revealed factorially In other contexts 
the same authors speak confidently about “auditory acuity,” “mechanical 
aptitude,” “fluency,” “visualizing,” and many other components of abilities 
and disabilities; but, when these same functions appear m a factor analysis, 
they promptly discard the psychological interpretations as foolish For such 
authors an attempt factorially to identify abilities is supposed to be a foi - 
bidden faculty psychology 

Statistical criteria for the reference frame 

If the principal purpose of a factorial study is to condense the intei con c- 
lations into a factor matrix with a set of numbers that will account for the 
conelations, then we have a legitimate statistical problem that can be 
solved In general, the factorial solution would be considered the more suc¬ 
cessful, the smaller the number of factois that account for the conelations 
with negligible residuals. The statistical condensation is judged to be suc¬ 
cessful if the number of independent parameters m the factor matrix is very 
much smaller than the number of independent correlation coefficients. By 
the usual statistical considerations the most acceptable factor matrix would 
then represent a reference frame that is called the principal axes of the con¬ 
figuration of test vectors It is located by a maximum variance criterion 
Some of the characteristics of the principal-axes solution will be mentioned 
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here. The fiisfc axis is so placed m the configuration that the sum of the 
squares of the projections of the test vectors on the fust axis is maximized 
There usually exists a unique solution for a given test battery. When the 
first axis is so placed, the first-factor products account for more of the vari¬ 
ances of the n tests than can be accounted for by any other axis in the con¬ 
figuration. When the same criterion is applied to the first-factor residual 
coefficients, the second pnneipal axis is so placed that it accounts for more 
of the residuals than any othei axis in the residual configui ation The 
fiist two factors, then, account for more of the test variances than any other 
pair of axes m the test configuration For any given numbei of factois so 
extracted from the correlation matrix, the residuals are minimized. This is 
an interesting and important feature of the principal axes of a test configu¬ 
ration This reasoning may be applied to a correlation matrix with unity 
or with communahties in the diagonal cells 

When the con elation matrix is large, say forty or fifty tests, the prin¬ 
cipal-axes solution is quite laborious, and it is usually advisable to make 
some compromise Anothei statistical principle that can be used for locat¬ 
ing the reference frame is to place the first axis through the centroid of the 
termini of the test vectors for the whole batteiy This axis is usually nearly 
the same as the first principal axis when the correlation matrix has coeffi¬ 
cients that are all positive or zero This type of solution can also be made 
either for the total test space or for the common-factor space. With some 
modifications, the subsequent axes can be placed so as to accomplish es¬ 
sentially the same purpose for the residual coefficients after the extraction 
of each factor This principle does not lead to a unique solution for any 
given test batteiy. Its purpose is essentially similar to that of the principal- 
axes solution, in that one attempts to account for as much as possible of the 
test variances by each successive factor; but the centroid solution is not 
unique for any given test battery, and the solution does not have the in¬ 
teresting mathematical properties of the pnncipal-axes solution The cen¬ 
troid method of factoring and the centroid solution for the location of the 
reference axes are to be regarded as a computational compromise, in that 
they have been found to involve much less labor than the principal-axes 
solution. The residuals after the extraction of a given numbei of factors are 
somewhat laiger than the residuals remaining aftei the same number of 
factors determined by the principal axes 

The two piinciples that have been descnbed for locating the refeience 
fiame are both dependent on the whole configuration Each successive fac¬ 
tor is determined by some criterion that is applied to the whole test con¬ 
figuration These criteria we have called "statistical” in contrast with the 
configurational criteria for locating the reference frame which depend on the 
nature of the test configui ation as determined by its distinguishable parts 
When we apply statistical critena for locating the leference frame, our 
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principal interest is in the size of the residuals, which are made as small as 
possible Less interest is then ordinarily given to the factor loadings them¬ 
selves, although some writers make attempts to interpret the factorial axes 
located by these criteria When the successive factorial axes have been lo¬ 
cated with the purpose of minimizing the residuals, the factor loadings 
are regarded more or less as regression coefficients, whose principal useful¬ 
ness is that they account for the dependent variable with the smallest 
possible residuals In most cases when a regression equation is written 
the principal concern is to predict the dependent variable with the small¬ 
est possible error, and little attention is then given to the interpreta¬ 
tion of the regression coefficients themselves They serve ordinarily as a set 
of useful, but otherwise meaningless, numbers that do predict the dependent 
variable In faetoi analysis we have no differentiation between independent 
and dependent variables, but we can apply statistical criteria to ascertain 
whether the several factors do account for the mtei correlations with mini¬ 
mum residuals If this is the puncipal purpose of a factorial study, then it 
seems reasonable to expect the writer to show that he gams something which 
is not available in the classical statistical methods involving multiple coi re¬ 
lation and regression equations 

The statistical and the configurational cntena foi locating a refeienee 
frame in the configuration of test vectois can be combined in what seems 
to be the ideal factorial study If we disregard computational labor, which 
may be reduced by impiovcments in computational methods, the ideal solu¬ 
tion for a factor problem seems to be, first, to deteiimne a set of r factors 
for the given correlation matrix which are so determined that the rth residu¬ 
al coefficients satisfy two requirements, namely, (1) that the residuals are 
the smallest possible for r orthogonal factors and (2) that they are small 
enough to be ignored This implies the first r principal axes of the test con¬ 
figuration When the solution has arrived at this point, we can be certain 
that the r factors account for more of the common-factor variances than 
can be accounted for by any other set of r orthogonal factors The factor¬ 
ing should be made with communalities m the diagonal cells of the correla¬ 
tion matrix The next step m the solution is to find a reference frame in the 
r-dimensional configuration which will represent the distinguishable func¬ 
tions or parameters of the tests and their correlations When such a refer¬ 
ence frame can be found, it represents primary factors in the test variances 
There is no guarantee beforehand that such a reference frame can be 
found If the test battery is a hodgepodge of complex tests, there may not 
exist m the configuration any clues as to what the underlying functions may 
be, but this possibility is dependent on the intuitions of the experimenter 
in assembling a significant set of measurements rather than on factorial 
theory and method as such 

In this text it is assumed that the principal puipose of factor analysis is 
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to identify one 01 more underlying factors, functional unities, or parameters 
that produce the differential performances or other individual diffeienees, 
even if the discovery of such a factor or factors should account for only a 
pait of the test variances in any pai ticulai test battery, leaving considerable 
parts of the variances still unaccounted foi and to be identified by futui e 
investigation 

If it should be found, foi example, that the ability for serial learning is 
distinct from the ability to memorize paired associates, then such a finding 
would be an impoitant addition to oui knowledge about the faculty or facul¬ 
ties of memory, even if the serial-learning factor should account for only a 
pait of the variances of the memory tests involved If it should be found 
that serial learning and paired associates do not sepaiate factonally, then 
we should conclude that, as far as we can ascertain now, these two forms of 
lecall are mediated by the same cortical functions The question of whether 
these factois account foi all, or only a pait, of the variances of the paiticu- 
lai tests that we happen to use is, then, a secondary matter for the scien¬ 
tific problem. At the same time, we should regard any unknown lesidual 
vananre as a challenge to fuither exploration either by factorial or by other 
expei imental means 

The paiameteis 01 functions in teims of which a domain is desenbed aie 
piobably never unique, and this corresponds to the fact that no one has 
evei demonstrated any factonal description to be unique The best that 
can be said for any factonal description of a domain is that it leveals some 
aspects of an undeilying order, but, in so far as this is successfully done, it is 
likely to leveal alternative sets of paiameteis 01 functions that might be 
chosen foi the scientific description In the simple two-dimensional example 
of the cylindeis in chapter u, this lack of uniqueness of the factorial resolu¬ 
tion is easily seen, but it does not invalidate either of the factonal desenp- 
tions The choice of fundamental axes would depend on the context, but it 
will be seen in the higher-dimensional problems that the common practice 
of taking a cential or aveiage factoi foi all the measurements in any particu- 
lai factonal expeiiment as one of the basic parameteis for the domain tends 
usually to blur rathei than to clarify the underlying order The lack of 
uniqueness of factoiial resolutions is m agreement with the punciple that a 
scientific hypothesis can never be demonstrated to be conect, it can only be 
shown to be plausible for the data at hand or inconsistent with them 

In the development of multiple-factor analysis one naturally raises the 
question as to whether the object is to account factorially foi the ongmal 
raw test scores or the mtercorrelations These two objectives are not the 
same One might repioduce the mterconelalions with communahties by a 
factoi matnx which would not enable us to repioduce the raw scores Some¬ 
times this question is discussed as an issue in which we might make a choice 
To analyze a set of raw scores so that they aie leproduced factoilally in- 
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volves more factors than can be solved for by present methods Limiting 
the analysis to the common factors sometimes makes the problem soluble 
By identifying one factor at a time and by building new test batteries for 
the exploration of unknown factors, we hope eventually to know enough 
about the factors involved so that the actual test performances themselves 
may be accounted for to an acceptable degree of approximation That is the 
ultimate objective Limiting ourselves to the common factoi s in each study 
is a means of eventually attaining the interpretation of the individual test 
performances 

Considering factor analysis as a scientific method rathei than as an effi¬ 
cient method of statistical condensation, we shall discuss different types of 
factors, configurations, and factor patterns to determine how they may be 
used in the discovery of an underlying order in the complexities of human 
traits or in other fields, wheie basic concepts may be formulated in teims 
of exploiatory factorial investigation 

Simple structure 

One of the turning-points m the solution of the multiple-factor pioblem 
is the concept of “simple structure ” It will be shown that this concept 
enables us to obtain an invariance of factorial description that has not, so 
far, been available by other means The combination of a test configuration 
and the co-ordinate axes is called a structure The co-ordinate axes deter¬ 
mine the co-ordinate planes, If each test vector is m one 01 moic of the 
co-ordinate planes, then the combination of the configuration and the co¬ 
ordinate axes is called a simple structure The corresponding factor pattern 
will then have one 01 more zero entries in each row If a test vector lies in 
one of the co-ordinate planes in a three-dimensional configuration, then it 
can be described as a linear combination of two co-ordinate vectors, so that 
one of its factor loadings is zero If a test vector lies m two of the co-ordi¬ 
nate planes m a three-dimensional problem, then it is collmear with one of 
the co-ordinate axes, and it will have two zero factor loadings If a test 
vector lies m all three co-ordinate planes m a thiee-dimensional problem, 
then it is a null vector with three zero factor loadings, and it is of no scien¬ 
tific interest. 

When a factor matrix reveals one or more zeros in each row, we can infer 
that each of the tests does not involve all the common factois that are re¬ 
quired to account for the intercorrelations of the battery as a whole. This 
is the principal characteristic of a simple structure. A simple structure and 
its corresponding factor pattern reveal that the complexities of the individu¬ 
al tests are lowei than the complexity of the battery as a whole When a 
configuration reveals that the test vectors lie m restricted subspaces, such 
as the co-ordinate planes, and in their intersections of still lower dimension¬ 
ality, this circumstance can be used for the location of a reference fiame 
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that will generally give a simpler interpretation for the individual tests than 
if each test is supposed to involve all the factors of the whole battery Most 
of the configuiations and factor patterns in the following section aie ex¬ 
amples of simple structure, in that generally the rows of the factor matrices 
have one or more zeros The degrees of convincingness and overdetercmna- 
tion of simple structuie and the algebraic relations will be discussed m sepa¬ 
rate sections 

Types of factors 

Factors aie classified in several types according to the manner m which 
they enter into the test variances of a battery. A factor which is of a given 
type m one battery may be of another type when it, enters into the tests of 
another battery. The classification refers, then, to the manner in which a 
factor is involved in a given test battery 

A factor which is involved m the variances of two or more tests of a bat¬ 
tery is called a common factor in that battery Our use of the term “com¬ 
mon factor” in this text implies that the factor is determined by the side 
correlations We distinguish between common factors so determined and 
the unique factors determined by the diagonal self-correlations Excep¬ 
tions to this use of the term will be explicitly noted 

A factor which is involved in the vananre of only one test of a battery is 
a unique factor m that battery 

A common factor which is involved in the vanances of all tests in a bat- 
teiy is a general factor m that battery 

A common factor which is involved in the variances of two or more, but 
not in all, of the tests of a batteiy is sometimes called a group factor m that 
battery 

A common factor which is involved m the variances of only two tests in 
a battery is called a doublet 

A common factor which is involved in the vananccs of only three tests 
in a battery is called a triplet 

A factor which enters into the test variances of a battery with both posi¬ 
tive and negative signs is a bipolar factor 

Types of configurations 

All the configui ations discussed in this section are three-dimensional 
The principles illustrated here can be geneialized to configurations m n 
dimensions 

Random configuration 

In Figure 1 we have three orthogonal axes, I, II, and III, which deter¬ 
mine the right spherical triangle shown The points represent normalized 
test vectois The test vectors in this figure show a random configuration, 



CONFIGURATIONS AND FACTOR PATTERNS 


183 


winch gives no clue as to factorial composition Hence the cells in the cor¬ 
responding factor pattern are, m general, all filled Both positive and nega¬ 
tive factor loadings are then found in the factor matrix This is illustrated 
in factor pattern 1 

Cone configuration 

In Figure 2 we have a cone configuration, in which the normalized test vec¬ 
tors all lie in a cone with a 45° generating angle The mtcrcorrelations for a 
cone configuration are all positive or zero So far, the cone configuration 
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Figure 1 

has been of only theoretical interest, and it is included here to bring out 
the fact that, although the correlations are all positive or zero, the load¬ 
ings for orthogonal factors cannot all be positive or zero This is illus¬ 
trated in the figure in which the cone configuration cannot be inclosed 
within the three orthogonal co-ordinate planes If the rank of the correla¬ 
tion matrix is 2 or 1 and if the correlations are all positive or zero, then a 
factor matrix can be written in which all the loadings are positive or zero 
When the rank of the correlation matrix is 3 or higher, then a positive fac¬ 
tor matrix does not necessarily exist. However, m practice it is usually pos¬ 
sible to write a positive factor matrix when the correlations are positive or 
zero because the test configuration is rarely, if ever, cone-shaped. 
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Isolated constellations 

In Figure 3 we have a set of three constellations or clusters of test vec¬ 
tors In Figure 3a the clusters arc shown at the co-ordinate axes, I, II , 
and III When the clusters are separated by acute angles, the clusters are 
con elated, and the co-oidmate axes would then be oblique if they were 
passed through the center of each cluster. The factor matrix 3d shows a 
simple structure with complexity 1 for each test, but, if the factois are cor¬ 
related, this circumstance is intei pi eted to mean that the three factors 
themselves have some second-order factor in common For example, visual 
and auditory acuity might be two factors in terms of which tests may be 
described, but these two factors might themselves be correlated by some 
second-order factoi, such as general physical fitness or age 


1 



In Figure 3b we have a different structure and consequently a diffeient 
factor pattern for the same configuration of three clusters Here axis I is 
placed in the middle of the configuiation, and one of the clusters has been 
placed in the plane 7-77 The other two clusters are then necessarily one 
positive on 777 and the other equally negative on III One cluster is a 
function of only two factors, namely, 7 and 77, while the other clusters in¬ 
volve all three factors Factor pattern 3b is not likely to reveal so much 
about the underlying functions as is pattern 3a. 

The isolated constellations or clusters constitute one of the most easily 
recognized of all the types of simple structure It can usually be seen by in¬ 
spection of the correlation matrix when such a configuration is present The 
tests within a cluster have relatively high mtercorrelations, while tests from 
diffet ent clusters have much lower mtercorrelations The differentiation of 
constellations was the first type of multiple-factor pattern to be investigat¬ 
ed, and it is the simplest form of configuration, since each test then has unit 
complexity 

Complete triangular configuration 

A three-dimensional configuration sometimes shows the test vectors to 
lie in three distinct planes, as shown m Figures and /fij In the simple 
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structure and factor pattern of Figure l±a there are both positive and nega¬ 
tive factor loadings because the configuration extends beyond the ti tangle 
l-H-IIl of the three co-ordinate axes In the simple structure of Figure 
4b there are only positive factor loadings because the test vectors aie con¬ 
fined to the positive light spherical triangle The test vectors m the plane 
l-II are described as lineai combinations of two factois, and the third- 
factoi loadings aie then zero It will be noticed that all the tests in the tn- 
angular configuration have a complexity of 2, except those which lie at the 
corners of the configuration, which have unit complexity. 
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Incomplete triangular configuration 

It does not usually happen in exploratory studies that all the co-ordinate 
planes are defined by the configuration In the three-dimensional case of 
Figure 5 we have a simple structure in which only two of the co-oi dmatc 
planes are defined by the configuration These are the co-ordinate planes 
I II and I—III If the fans of test vectors in the planes J —II and I-III 
extend over right angles and if the dihedral angle at I is a right angle, the 
location of the three orthogonal axes as shown would be fairly convincing, 
even though the plane II—III was not defined by the configuration. In 
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such a situation one would try to find an interpretation for the factois II 
and III to determine whether there is some good reason why they should 
not have combined m some of the tests so as also to produce a fan in the 
plane II-III New' tests might he constructed in which these two factors 
are involved and a new factoi study made to ascertain whether the inter¬ 
pretation of factors II and III can be sustained by the appearance of a fan 
of test vectors m that plane The present example includes only positive 
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factor loadings, but the same general type of reasoning can be extended to 
simple structure with negative factor loadings, as will be shown in other ex¬ 
amples 

If the dihedral angle at I is not a right angle, the inference is made that 
factois II and III are probably correlated, either positively or negatively, 
depending on their exact locations. If the fans of test vectors from the first 
axis do not extend to a full right angle, as shown in Figure 5b, then the loca¬ 
tion of the co-oidinate plane II-III is indeterminate It might be assumed 
to be orthogonal to I, or it might be placed in the oblique position II'—Ill ', 
as indicated by the test vectors in the two planes But such a situation may 






CONFIGURATIONS AND FACTOR PATTERNS 


187 


have arisen because the test battery did not happen to contain any test 
measurements that were functions of I and II neai II, or of I and III near 
III. This is a fiequent type of ambiguity m the factorial resolution of test 
batteries It is not profitable to insist that factoiial method as such should 
identify the locations of II and III m the configuration if these factois are 
not involved either separately or m some combinations in the batteiy It 
is better to make some interpretation of what II and III might represent 
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and then construct new test batteries involving these factois m the puiest 
possible form as well as in combination If the picdicted factoiial composi¬ 
tions are sustained by theii location in a defined plane II—III, the hypothe¬ 
sis can be accepted, but, if the new tests scatter in a diffeient manner, the 
hypothesis concerning the nature of II and III must be discarded Factoiial 
analysis then becomes a scientific method whereby proposed factoiial inter¬ 
pretations become questions of fact to be determined experimentally 

It should be noticed that in the interpretation of factorial results we 
speak of a hypothesis being sustained We do not say that it is pioved to 
be correct No matter how convincing any experiment may seem to be as 
legards some scientific hypothesis, it is always possible that future experi- 
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tnents may show us to be wrong, as the fundamental ideas of a science are 
modified or replaced by more embracing conceptions. What is a discovery 
of interest and importance in one geneiation becomes inadequate or even 
wrong in the next generation There is little use m belaboring this limita¬ 
tion, which is universal for all science These reservations will be taken for 
granted throughout our discussion of factor analysis as an aid in scientific 
exploration 
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General and group factors 

Since factor analysis was dominated for a quai ter of a century by the cen¬ 
tral theme of a single general intellective factor with other factois regarded 
as “disturbers” of secondary impoitanee, it is of interest to see perhaps the 
simplest form of geneial factor in a configuration One type of configura¬ 
tion involving a general factoi in a battery is shown in the structure of Fig¬ 
ure 6 The simple structure is incomplete m that only two of the co-ordinate 
planes are defined by the configuration, namely, I-II and I-II1 Note also 
that all the test vectors have positive projections on the first axis, so that 
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the first column of the factor pattern is filled All the correlations would be 
significantly positive The rank of the correlation matrix would be 3 

The plane 71-/17 is not defined by the configuration m the structuie of 
Figure 6a Some students of factor analysis prefer to locate the reference 
frame for a configuration of this type as shown in Figure 6b They place 
the first axis centrally m the configui ation, so as to maximize the projections 
of the first factor, as shown The location of orthogonal axes II and III 
can then be made by one of several statistical criteria The corresponding 
factor pattern is then, m general, filled as shown, the second and third fac¬ 
tors being both bipolar. The resolution of Figure 6a is the simpler and moie 
readily interpreted, but the first-factor saturations are not maximized 

When a configuration is found such as that of the two figures, 6a and 
6b, there is some uncertainty in the interpretation of all three of the fac¬ 
tors The second and thud factors may not exist in. the absence of the fiist, 
or they may be correlated with the first factor, in which case the structure 
would be oblique The first factor is defined by its presence in all tests of 
the battery, but it would be interpreted more clearly if we could predict 
with assurance the tests m which It is absent, as well as the tests in which it 
is present A new factor experiment should then be made with a variety of 
tests, in which it is predicted that the first factor should be absent but m 
which one or both of the other two factors should be piesent The simplest 
and clearest result would be to find a complete triangular structure with a 
gioup of tests m which the first factor is absent. If the predicted configura¬ 
tion is obtained, one has more assurance m the mterpietation of the factors 
than if the factorial result is left as in Figure 6 Whatevei the general factor 
is in a particular battery, its nature is better understood if we can predict 
with confidence the tests in which it is absent, as well as the tests m which 
it is present Since factor analysis was for so many yeais primarily con¬ 
cerned with the single genei al factor and with cori elation matrices of rank 1, 
the procedure here suggested is stijl avoided by some mvestigatois, even 
though it is indicated when the problem is regarded from the point of view 
of multiple-factor analysis 

Bipolar simple structure 

The concept of simple structure implies that the test vectors lie in the co¬ 
ordinate planes, or hyperplanes; but it does not imply that the factor load¬ 
ings are all positive, as this concept is often misstated. By way of example, 
Figure 7 has been drawn to show one bipolar factor and two positive factors, 
whose saturations are all positive or zero It is factor I, which here is bi¬ 
polar A configuration such as that of Figure 7 would be interpreted as 
showing that factor I combines either positively or negatively with III but 
that it combines only positively with II The figure shows only one of a 



190 MULTIPLE-FACTOR ANALYSIS 

great variety of configurations m which positive and bipolar factors can be 
combined, 

Dependent composite factor 

Occasionally, but not often, it happens in analyzing experimental data 
that the trait vectois define a secondary plane or hypeiplane, as shown at 
II-A in Figure 8 for a thieo-dimensional configuiation When a secondary 
plane like this appears m the configuiation, the interpretation is likely to be 
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Factor pattern 7 


FicmitE 7 

of special interest The test vectors in the secondaiy plane II—A can be de¬ 
scribed as linear combinations of I, II, and III, where I and III always en¬ 
ter into the tests m the same ratio The configuration of the figui e would 
mean that whenevei I and III combine with II, then I and III entei the 
measurement in some fixed ratio. The test vectois m the three bounding 
co-ordinate planes are all linear combinations of only two of the factors A 
situation like this should not be ignored when it is found m experimental 
c ata It may be suggestive for some fundamental concepts in the domain 
that is being investigated 
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The factor 'pattern 

The factor pattern has been defined as a factor matrix m which are re¬ 
corded the non-vamsliing cell entries denoted X and the vanishing entries 
which aie left blank When a differentiation is to be made between positive 
and negative factor loadings, the non-vanishing entries are denoted -j- or —, 
while the vanishing entries are blank For the inteipretation of factonal re¬ 
sults the factor pattern is sometimes of greater importance than the numen- 
cal values of the factor loadings. It is sometimes important to know about 
an unknown factor or function that it is present in certain tests and absent 
m certain others, or that it enters positively (as m a numeratoi) m some tests 
and negatively (as in a denominator) in certain other tests In fact, it is 
more significant to know the factor pattern m this way than it is to know the 
actual numerical values of the factoi loadings, because the numerical values 



Dependent composite factor 

are determined by such melevant matters as the metric, the numerical 
scales in terms of which the measurements were made, and the decision of 
the investigator to reduce the measurements to correlations 01 to leave them 
in the form of covariances It will be shown that ceitam fundamental foims 
of invariance in factor analysis relate to the mvanance of the factor pattern. 
Not all these forms of invariance aie applicable to the numerical values of 
the factor loadings 

The factor pattern can be regarded m two different ways dunng a factoi 
analysis It can be set up as a hypothesis at the beginning of an analysis, 
and the factoring may be earned out with a specified pattern in mind, or the 
factor pattern can be inferred from the configuration when the factoring has 
been completed Some wiiters postulate a pattern m which the fiist factor 
shall be positive, leaving the subsequent factors bipolar It is piobably 
better to factor a correlation matnx and then examine the configuration 
before choosing an appropriate factor pattern. The examples of this chapter 
serve to illustrate the relations between the configuration, the choice of 
leference frame, and the resulting factor pattern The principles illustiated 
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in these examples can be generalized to configurations in any number of di¬ 
mensions 

Some of the controversy about the general factor is caused by the insist¬ 
ence of some writers that there must be a geneial factor, even if it is only an 
average of the test vectors in a battery whose correlations are positive or 
zero In order to illustiate this situation m three dimensions, we have drawn 
Figure 9, which shows a set of test vectors m a complete triangular configu¬ 
ration When these are plotted on a sphere, the location of the reference 
frame, as in Figure 9a, is compelling and convincing Instead of placing 
the reference frame in lelation to the configuration, some writers place the 
first axis at 7, as shown in Figure 9b, because this location is central for the 
whole battery and maximizes the test vector projections The other axes 




Alternative locations of first reference axis 

are placed orthogonal to the fiist central or average axis, and they take posi¬ 
tions like II and III outside the configuration so that they necessarily be¬ 
come bipolar It is still possible to identify the more fundamental factors, 
A, B, and C, after a central axis has been placed in the configuration be¬ 
cause, as is to be expected, the first-factor residuals will be largest for the 
tests near the corners of the triangle, and these are the tests that primarily 
identify the factors One type of factorial interpretation that is rather com¬ 
mon can be illustrated in Figure 9b The thud factoi would have positive 
saturations on the tests near B and C, while it would have negative satura¬ 
tions on tests near A Such a result is sometimes inteipreted to mean that a 
factor such as III in Figure 9b is a “complex relationship” of some kind 
If these writers could only be induced to plot their configurations and look 
at them, there would be less insistence on placing the fiist axis centrally in 
the configuration, and the factors would often yield to rather simple psy- 



The Mots would more often appear as understandable psychological en¬ 
tities or fictions When a bipolar factor appears in which the positive 
saturations form one or more meaningful groups and the negative satura- 




CHAPTER X 
ROTATION OF AXES 


Oblique axes 

In previous cbapteis we have considered the uitation of axes in configuia- 
tions that could be completely visualized on a diagram in two dimensions 
or on the surface of a spheie in thiee dimensions In actual data the dimen¬ 
sionality neaily always exceeds three, so that the configuration cannot be 
completely visualized m a single diagram or model The rotation of axes 
is then determined by two-dimensional sections, one section foi each pair 

Table 1 

A Centroid Factor Matrix Fo 



I 

II 

ill 

A 

B 

c 

1 

659 

- 736 

138 

2 

726 

180 

- 656 

3 

665 

537 

500 

4 

869 

- 209 

- 443 

5 

834 

182 

508 

6 

836 

519 

152 

7 

856 

- 452 

- 269 

8 

848 

- 426 

320 

9 

861 

416 

- 299 

10 

880 

- 341 

- 354 

11 

889 

- 147 

.436 

12 

875 

485 

- 093 

13 

667 

- 725 

.109 

14 

717 

246 

- 619 

15 

634 

501 

522 

16 

936 

257 

165 

17 

966 

- 239 

- 083 

18 

625 

- 720 

166 

19 

702 

112 

- 650 

20 

664 

536 

488 


of co-ordinate axes, 01 by thiee-dunensional sections of the multidimension¬ 
al configuration A four-dimensional configuration is repi esented by six dia- 
giams when two-dimensional sections aie used A configuration of ten di¬ 
mensions has forty-five such sections, since this is the number of pairs of 
co-ordmate axes 

In this chapter we shall use as an example a three-dimensional configura¬ 
tion with twenty test vectors This example is used for illustrating several 
methods of rotating the axes, namely, (1) by plotting the normalized vec- 
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tors on a spheie, as shown m a previous chapter, (2) by two-dimensional 
sections, as will be shown m this chapter, and (3) by three-dimensional sec¬ 
tions in the method of extended vectors, which will be described m chap¬ 
ter xi. 

We start this problem with a factor matrix F a in which n — 20 and r = 3, 
as shown in Table 1 The columns of this matrix represent three arbitraiy 
orthogonal axes as determined by the factoring method These thiee axes 
are denoted I, II, III, and they will be referred to by the subscript m, as in 
previous chapters These arbitraiy oithogonal axes, as well as the test con- 


4 



figuiation, will be legalded as fixed in position, and our problem is to select 
by successive approximations another set of axes for which we hope to find 
scientific interpretation The new set of axes will be called reference axes, 
and they will be denoted by the letters A, B, C, etc , to distinguish them 
from the fixed orthogonal frame I, II, III. 

In starting the rotational problem we locate the unit reference vector A 
in the orthogonal axis I, the reference vectoi B in the orthogonal axis II, 
and the reference vector C in the orthogonal axis III This is shown in 
Table 1 and in Figure 1 The initial location of the reference vectors will be 
denoted Ai, ff, C\, and they will be identified by the subscript u Their 
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direction cosines are stiown in the matrix Aoi with elements X mu , as shown 
in the rectangular notation, 


I 

II 
III 

Aoi with elements ^niu 

The subscripts of Am show that the columns of this matrix give the direc¬ 
tion cosines of the initial locations (1) of the unit reference vectors with re¬ 
gard to the fixed orthogonal frame (0) The subscripts of X m „ refer to the 
axes of the fixed orthogonal frame (m) and the given positions of the ref¬ 
erence vectors (it). 

For each rotation we denote the given positions of the leferenoe vectois 
Ai, Bi, and Ci. By inspection of the configurations in Figures 1,2, and 8, we 
determine the adjustments to be made m these vectors Their new posi¬ 
tions are denoted A%, f? 2 , and C 2 After the first rotation the plots repie- 
sent oblique axes, so that the true lengths of the vectors are not immedi¬ 
ately evident. The adjusted vectors aie therefore first determined as to 
their direction They are then called long reference vectors and denoted A 2 , 
B’ 2i and C 2 When they have been reduced to unit length, they constitute 
the new unit reference vectors, A 2, B%, and C\ For each adjustment of the 
reference vectors we have, then, two steps in computation Starting with 
the given unit reference vectors Ai, B 1 , Ci, we determine, first, the long 
reference vectors Ai,, B[, C 2 , which give the desired directions Second, we 
determine the lengths of these vectors, so that they can be reduced to the 
new unit reference vectors As, Bs, C2 

The projections of the test vectors on these axes are then computed, and 
a new set of plots is made By inspection of these plots the next adjust¬ 
ments are determined by inspection In the computations for the second 
rotation the given unit reference vectors are denoted A u B u C u and their 
newly adjusted positions are denoted A 2 , B 2 , C' 2 In this manner we avoid the 
use of numerically large subscripts, say 10 or 15, for as many rotations. 
For each rotation of the reference axes, the given positions are indicated by 
the subscript 1 and the new positions by the subscript 2 

The given reference vectors will be indicated by the subscript u The 
new long reference vectors for each rotation will be indicated by the sub¬ 
script r, and the new unit reference vectors by the subscripts p and q 
In Figure 1 we have plotted the first two columns, and each point is 
labeled with the number of the corresponding test vector. The points in 
this figure show the configuration of test vector termini as they would ap¬ 
pear when seen along a line of regard orthogonal to the plane of A 1 B 1 The 


Ai Bi Ci 
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tw> axes A i and B ; are m the plane of the diagram If the axis Ai is moved 
in the plane A 1 B 1 to the direction of A' 2 , it will deteimme a plane (or a hy- 
peiplane if r> 3) which will intersect the plane A l Bi in the line which is 
marked A-plane This line is a trace of the plane, or the hyperplane The 
vectors 1, IS, and IS will then have vanishing projections on the vector A[ 
In the same manner the given position of the axis B, which is denoted B i, 
can be moved in. the plane AiB, to the direction of B' Orthogonal to this 
vector is the Bplane, as shown When the axis B, is so moved, theie will 
be a group of test vectors with small oi vanishing projections on B' 2 

In Table 2 we have summarized the computations for rotating the refei- 
ence axes Each step m the computations will be described in detail for the 
first rotation, but the computations will be summarized only for the subse¬ 
quent rotations The purpose of the rotations is to maximize the numbei of 
test vectors with zero piojections on the reference axes The given vectors, 
Ai, B i, and G i, are repiesented by the rows of the 3X3 matrix, S i 2 , with 
elements s U r The three columns represent the new long reference axes, 
here denoted A,, B' v C 2 The vector Ai is to be changed to the vectoi A 2 , 
which is oithogonal to the A-plane The vector A( is a linear combination 
of Ai and Bi It can be expicssed in the foim 

(1) Aj = 1 OOAi + ,90J5i, 

as read dnectly from the diagram of Figure 1. 

In the same figure we determine B 2 , which is normal to the trace, 
B-plane, as a linear combination of Ai and B, It is read graphically as 

(2) B' 2 = 50Ai-100Bi. 

The dnections of A 2 and of Bj are so chosen that the majority of the 
twenty points have positive projections 

The coefficients of these two equations are enteied in the columns of 
matrix Su of Table 2 The columns of that matrix show the direction num- 
beis of the new vectors as linear combinations of the given vectors Two of 
the columns of that matrix aie determined graphically from Figure 1 
In Figure 2 we have plotted the given columns Ai and C 1 from Table 1 
In this figuie we see the opportunity to move the vector C 1 to C' 2I orthogo¬ 
nal to the C-plane, whose trace passes through a group of points The new 
vector can be expressed as a linear combination of A x and C 1 , namely, 

(3) C' = 40 A 1 - 1 00C 1; 

as determined graphically, and the coefficients of this equation are entered 
in the third column of the matrix »Si 2 of Table 2 Here, again, the new vec¬ 
tor Cl is diawn in such a direction that the majonty of the points have 
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4<u5i* = Lot 
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Table 2 


Rotation 1 



A 1 

B', 

Ci 

Ai 

1 00 

50 

40 

Ri 

90 

-1 00 

— 

Ci 

— 

— 

-1 00 


1 90 

- 50 

~ 60 



AS 

B’i 

cl 

I 

1 00 

50 

40 

II 

90 

-1 00 

— 

III 

— 

— 

-1 00 

2 P 

1 81 

1 25 

1 16 

Vj-p 

1 3454 

1 1180 

1 0770 

dr p 

7433 

8944 

9285 



Ai 

Bi 

Ct 


743 

447 

371 

Si 

669 

- 894 

— 

Cl 

— 

— 

- 928 


1 412 

- 447 

- 557 


Table S 


Factor Matrix F 0 A 02 = V 2 



A 2 

Bi 

C, 

1 

- 003 

953 

116 

2 

659 

163 

878 

3 

853 

- 183 

- 217 

4 

506 

575 

734 

5 

741 

210 

- 162 

6 

968 

- 090 

169 

7 

334 

787 

567 

8 

345 

760 

018 

9 

918 

013 

597 

ID 

426 

698 

655 

11 

562 

529 

- 075 

12 

.975 

- 042 

411 

13 

Oil 

946 

146 

14 

697 

101 

840 

15 

806 

- 164 

- 249 

16 

867 

189 

194 

17 

558 

645 

435 

IS 

- 017 

923 

078 

19 

597 

214 

864 

20 

852 

- 182 

- 207 

Ch 

11 655 

7 043 

5 792 

X 

11 655 

7 045 

5 792 


LotDi =» Am 

At Bt Ci 

1 

XI 

III 

743 447 371 

669 - 894 — 

— — - 928 

s 

1 412 - 447 - 557 


A oa \o2 — Cj 

Ai Bt C'i 

At. 

1 VQ Bi 

Ci 

1 000 

- 266 999 

276 166 999 
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positive projections on (X Figure 3 was plotted from columns B and C of 
Table 1 but this figure was not used in this rotation of the reference frame 

The first step m the rotation of the reference axes is to determine graphi¬ 
cally the numerical entries m the matrix <Su with elements s ur The sub¬ 
scripts of S 12 indicate that this matnx shows the adjustments m the ref- 
etence vectors from position 1 to position 2 The elements s ur represent the 
relation between the given unit reference vectors, u, and the new long ref¬ 
erence vectors, r 

In older to determine the tiue lengths of the long reference vectois Al,, 
B' 2 , On, they should be referred to the given orthogonal reference frame 



Figure 2 


This frame is actually represented in A h B h and C i of the first thiee fig- 
uies, but it is not so directly lepresented in the diagrams for subsequent ro¬ 
tations Let a matrix L 0 2 , with elements l mT , represent the long reference 
vectors r in terms of the fixed oithogonal frame m The subscripts of Lm 
indicate that the long reference vectors in the second position are expressed 
in terms of the fixed orthogonal frame (subscript 0) 

The columns of the matrix A 01 with elements X m „ show the direction co¬ 
sines of the reference vectois in their positions 1 Since the initial positions 
of the refeience vectors are m the fixed orthogonal fiame, we have A 1 in 
the axis I, B, in the axis II, and C, m the axis III, so that A 01 is an identity 
matrix The elements of the matrix A are denoted when it represents a 




200 


MULTIPLE-FACTOR ANALYSIS 


set of reference vectors before adjustment The elements of A are denoted 
X mp when it represents the reference vectors after an adjustment has been 
made. 

Let A 02 be a matnx with elements \ mp whose columns show the direction 
cosines of the leference vectors in their second positions The columns of 
this matrix show the direction cosines of the unit refeience vectois p m the 
second position, expressed in terms of the fixed orthogonal frame m 



Figure 3 


In order to dcteimine the true lengths of the long reference vectors A.'„, 
Z? 2 , and C' 2l we describe them in terms of the fixed orthogonal frame We 
then have 


(4) 



Imr 


winch can be written in the rectangular foim 
(5) 



A\ 

B, 

C\ 


A 2 

Bi 

Gi 

= 

Ai 

Bi 

Ci 
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A ! 

1 00 

SO 

40 

I 

1 00 

50 

40 

II 


1 


b l 

90 

-1 00 

— 

II 

90 

-1 00 

— 

III 



1 

c, 


— 

-1 00 

III 

— 

— 

-1 00 


^77,r 
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In the matrix \ mu we have the vectors Ai, B 1 , and Ci expressed m terms of 
the orthogonal unit reference vectors I, II, III In the matux s u , we have the 
new vectors A?, B' 2 , and C 2 expressed m terms of the given vectors At, B 1 , 
and C\ The matrix multiplication gives the product matrix with elements 
l mr , in which the long reference vectors A', lt B' 2l and Ci, are also expiessed in 
teims of the fixed orthogonal unit referenoe vectors The matrix \ mu is a 
unit matiix for the fiist lotation Consequently, s„ r and l mr are heie iden¬ 
tical, but they will diffei for subsequent rotations. 

The matrix equation can also be written m the form 

(6) Aol<Su2 = A 02 , 

where the columns of L a 2 show the dnection numbers of the long lefeience 
vectors in position 2 in terms of the fixed orthogonal frame 

Since the columns of L 0 2 in Table 2 show the projections of the long vec¬ 
tors on the fixed orthogonal reference frame, we can determine the true 
lengths of the vectors A' 2 , B 2 , and C 2 The sum of the squares in each col¬ 
umn is written 




or, more briefly, as 2 l 2 on the data sheets The square root of each sum is 
VsT 2 , and it is the length of the vectoi r. The reciprocal is 


(7) 


d, p = 



The values of d rp may be regalded as the elements of a diagonal matrix D 2 . 

The next step in the computations is to reduce the long reference vectors 
to unit length This is done by the relation 


( 8 ) 




rdrp X„ 


or, in matrix foi m, 


(9) 


A02-D2 — A .02 , 
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which can be written m rectangular form 



The oblique factor matrix which is the result of this rotation can now be 
computed by the equation 

(11) OfjmFmp Vjp 
or, m matrix form, 

( 12 ) FoAm = Vi i 

where F o is the given factor matrix and F 2 is the oblique factoi matrix This 
is the factor matrix in its second position If the rotation of reference axes 
were limited to orthogonal transformations, we shonld have 

(13) F oAa2 = Fz > 

where F 2 would show the projections of the test vectors on a new set of 
oithogonal axes 

If it is desired to compute each factor matrix V from the just preceding 
factor matrix, instead of making the computations at each rotation from 
the original orthogonal factor matnx Pa, then we use the transformation 
matrix h„ P instead of \ mp The columns of the matrix H with elements h up 
show the new unit reference vectors A 2 , B 2 , and C 2 m terms of the preceding 
reference vectors A,, B h and C\ We then have 

(14) St tr d rp hup 

r 

or, m matrix form, 


( 15 ) 


S12D2 — Hu 
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or, in rectangulai form, 

(16) 

S 12 Dz U 12 



Ai 

Si 

Ci 


A.2 

S 2 

C, 

= 

A.1 

B , 

C 2 

A, 

1 00 

50 

40 

A[ 

743 



Ai 

743 

447 

371 

Bi 

90 

-1 00 

— 

Bi 


894 


Bi 

669 

- 894 

— 

Ci 

— 

— 

-1 00 

Ci 



929 

Ci 

— 

— 

- 928 


Sue rfrp “'tip 


The oblique factor matrix IT can be deteimined from the preceding 
factor matrix V i by the transformation II n We then have 

(17) ViHn = IT 

The rotational problem starts with the factor matux F 0 with elements 
a im , which give projections of the test vectois j on the fixed orthogonal 
frame m The first step is merely to assign the initial locations of the tnal 
reference vectors in the orthogonal axes of the fixed frame This is repre¬ 
sented by the transformation 


FuAui = IT, 

m which Aoi is an identity matrix The first rotation lepresents the inde¬ 
pendent adjustments of the reference vectors from their initial positions in 
the fixed axes This adjustment is lepiesentecl by the equation 

F ah. m = V 2 , 

in which the columns of A 0 2 give the direction cosines of the new refeience 
vectois in terms of the fixed orthogonal vectors m The axes of V 2 are then, 
in general, oblique 

The second rotation is represented by the equation 

FoAoa = IT , 

where the columns of Aoa show the dnection cosines of the adjusted refer¬ 
ence vectors The oblique factor matrix V 3 can be obtained from the next 
preceding factoi matux, IT, if desired The relation then is 


TTH 23 = T 3 . 
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The cosines of the angles between the new refei ence axes should be de¬ 
termined, to make suie that no pair of them is approaching anothei pair so 
closely as to represent piactically the same factor If that should happen, 
the dimensionality of the oblique factor matrix would be reduced so that 
the columns would be linearly dependent It would then be necessary to alter 
the location of one or more of the leferenee axes With a little practice this 
error can be avoided The cosines of the angles between the reference vec¬ 
tors aie determined by the cross products 


( 18 ) 




PI 


or, in matrix form, 

(19) A'A = C . 

After the first lotation we have 

(20) Ao 2 Ao2 — Ci . 

It is useful to record these cosines on the corresponding diagrams for the 
next rotation, so as to avoid moving any pair of reference vectors so close 
together that a dimension is lost The diagonal values of A'A must be unity, 
since unit vectors are involved 

The successive steps in the computations for the first rotation are shown 
in Table 2, and they can be summarized as follows 

1) Write the elements of the matrix Su by inspection of the plots foi 

Fi 

2) Pei form the matrix multiplication 

(6) AoltS'i2 = L 02 , 

where Am is a unit matrix for the first lotation 

3) Determine the stretching factors d rv from the sums of squares m col¬ 
umns of Lq 2 

4) Compute elements X OTJ> of A 02 by the equation 

(9) Z/ 02 D 2 = A 02 • 

5) If it is desired to compute the oblique factor matrix, Vi, from the next 
preceding factor matrix, V\, then compute the elements h up in the matrix 
H 12 by the equation 


(15) 


SnD s = Hn . 
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G) Determine the cosines of the angulai separations of the new unit lefcr- 
ence vectors by the equation 

(20) A 02 A 02 = C 2 

7) If these cosines are acceptable, compute the new oblique factoi ma¬ 
trix, Vi, by either of the two following equations. 

a) In terms of the original factoi matrix, we have 

( 12 ) FaK.ni = Vi. 

b) In terms of the preceding factor matrix, we have 

(17) V 1 Hu=V 2 . 

In the first rotation the initial factor matrix, V is identical with F 0 , which 
is the initial factor matrix obtained from the experimentally determined 
correlation matrix The matrix V 2 is shown m Table S 

8) Plot a new set of diagrams for all pairs of columns of V 2 and examine 
them to determine the adjustments, which will be the next rotation of 
axes. 

It may be helpful to visualize in a single table the computations that are 
summarized in Tables 2-7, inclusive, for the several rotations of the refer¬ 
ence axes m the present illustrative pioblem The formulae have been ar¬ 
ranged in order of computation and in a separate column for each rotation 
The subscripts indicate the rotations to which the formulae belong The 
same schema can be extended, of couise, to any number of lotations 

Computational formulae for three rotations 

Given the factor matrix F a —Vi as obtained from the correlations and 
the identity matiix Am. 


Rotation 1 

Rotation £ 

Rotation S 

Determine Su giaphically 

Determine >S 23 graphically 

Determine S 34 graphically 

A01S12 = Ij 02 

Aq 2$23 = -Do 3 

Aq 3&34 = Lq 4 

D2 from I/02 

.D3 from Lo3 

Da from Lot 

[1S12D2 = iJia] 

[#23-0 3 = H22] 

= Hu] 

L02D2 = A 02 

L03D3 = An 3 

LoaD 4 — A04 

A02A02 — O2 

AoaAoa = C&. 

A04A04 = @ \ 

[ViHn = 7 2 

( V2H23 ~ V a 

fe¬ 

ll 

fcj 

n 

fe-_ 

| or 

< or 

< O? 

[ ^ 0 A 02 = V2. 

i/^oAoa = V3 

fe¬ 

ll 

4 

0 

fe. 

(In first rotation these two 

(These are two altei native 

(These are two alternative 

formulae are identical) 

formulae) 

formulae) 

Plot pairs of columns of V2 

Plot pairs of columns of V 3 

Plot pairs of columns of V 4 
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It should be noted that the matrix pioduct of the seveial transformations 
H is equal to the transformation A foi the same rotations We then have 

(21) Ii ai.H nH isH zi = A 01 , 

where the transf 01 mation H 0 x = Ao 1 = / The transformation h up carries a fac¬ 
tor matrix V n —1 to the next factor matrix V n , while the transformation Ao* 
carries the original factoi matrix Fa to the oblique factoi matrix V n In gen¬ 
eral, the computational labor is least when the oblique factor matrix is com¬ 
puted by the transformation H, but if a matrix-multiplying machine is avail¬ 
able, the computations are performed best by the ti ansformation A 



The result of the first lotation of axes is shown in the next thiee figures 
(4--6) : where pairs of columns of V 2 are plotted as before. It should be noted 
that, while the reference axes are now oblique, they are nevertheless ‘plotted on 
orthogonal co-ordinate cross-section paper, as indicated m the figures The 
procedures here described do not require that the plots be actually made 
with oblique axes on the graphs 

In stating each new rotation we denote the previous axes with the sub¬ 
script 1 and the new axes with the subscript 2 In Figure 4 we have plotted 
the columns A and B of Table 3 The vector B 1 is moved to the position 5(, 
so that the trace of the If-plane goes through a group of points 3, 15, 20, G, 
9,12 The new vector B' 2 is described from the graph by the vector equation 

(22) B! 2 = 1 005 1 + lOAi. 
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In Figure 5 we have plotted the columns A and C of Table S. A 1 adial 
tiaee for a new A-plane can be drawn, as shown, with the normal A£. Such 
a move of the A-axis will not only retain the zero, or near-zero, projections 
of 1 , 13, and 18, which are at the origin, but will add six more testa to the 
list of negligible or zero projections on the A-axis, The new reference axis 
A' 2 can be expressed as a linear combination of the axes A i and C i of Figure 5 
This relation is 

(23) A' = 1 00A! - .72(71. 

In the same diagram we see the opportunity to move the (7-plane so as to 
obtain zero projections not only for those tests which are shown at the 



Figure 5 


origin but also for the group 3, 5, IS, 20. The normal to the (7-trace is C 2 , 
which can be written in the form 

(24) C£ = lOOC'i + 25Ai. 

In Figure 6 the trace of the C-plane is drawn radially so as to include the 
group of tests 1, 13, 18, and its normal is C' 2 , which is written 

(25) C'r, = 1 00<7i — .12_Bi. 

In the same diagram the trace of the S-plane is drawn radially so as to pass 
through the group 2, 14, 19, and the normal is written 

(26) S' = 1 00S, - 


20(7i 
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The numerical values in equations (13)-(15) are entered in the ^-matrix 
of Table 4 Here it will be noted that a new process has been introduced, 
namely, the combination of movements of a plane from more than one dia¬ 
gram The new vector B' 2 is determined here as a sum of three vectois, 
namely, the previous B vector plus two corrections, namely, + 10A l5 and 
— 20C'i In a similar manner, the new vector C£ is determined by two cor¬ 
rections added to the previous vector C, namely, +,25Ai and — 12£i, as 
shown in the columns of S 13 in Table 4 

In combining several corrections m the same rotation it is useful to re¬ 
member that the corrections should not be large—say, not over 30 If a 
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Figure 6 


rotation whose sine is greater than about 30 is indicated on a diagram, it 
should be taken singly for that rotation If several small rotations are indi¬ 
cated on the diagrams, they may be combined if they are independent The 
independence of two small rotations can be seen in the present examples 
The new vector Cj is determined by two corrections from Figures 5 and 6 
In Figure 5 the trace of C is moved toward the group 3, 5, IS, 20, and these 
are near the origin in Figure 6. In Figure 6 the trace of C is moved toward 
the group 1,13,18 , and these are near the origin in Figure 6 Hence the two 
movements can be combined If a trace can be moved towaid a certain 
group of tests in several diagrams, the move is made on only one diagram, 
for otherwise there would be an overcorrection Another useful principle is 
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that if a trace moves toward the configuration in one diagram (as C in 
Figure 6) and away from the configuration m another diagram (as C in. 
Figure 5), then the two corrections can be added in the same rotation with¬ 
out checking the numbers of the variables 

Another example of the combination of two corrections is the 5-traee 
in Figures 4 and 6 Here, again, one of the movements is toward the con¬ 
figuration (Figure 6) and the other is away from the configuration {Figure 4). 
The two movements are written as follows • 


(27) 

5' = 1.005! - 

•20Ci, 

and 



(28) 

5' = 1.00#! +• 

.lOAt. 


The two corrections are shown in the iS^-matrix of Table 4- 
The steps in the computations of Table 4 are as follows • 

1) Compute the elements of L 0i by the relation 

A05/S23 = T03 * 

2) Compute the diagonal elements d, v of the diagonal matrix D s 

3) Compute the elements of the matrix A03 by the relation 

L03H3 = A03 . 

4) Compute the cosines c vq of C 3 by the relation 

A03A08 — C 3 . 

5) Compute the elements of the next factor matrix by the relation 

FoAos — V 3. 

6) Plot pairs of columns of Vs on orthogonal cross-section paper and re¬ 
cord values of c P5 on each diagram. 

The computation of is optional as an alternative method of arriving 
at V 3 If it is desired to compute Vs from Vi, then the computation is made 
by the formula 

V iHis — V 3 

If it is desired to compute V 3 from the original factor matrix F 0 , then we use 
the formula 

F 0 A 03 — V 1 • 

In any event, the latter formula should be used for check on computations 
after five or six rotations have been made by the transformation H. 

The three plots for Ya of Table 5 are shown m Figures 7, 8, and 9 These 
plots show that the reference axes are nearly in their final positions m the 



Figure 7 



COl - - / 77 


Figure 8 









212 


MULTIPLE-FACTOR ANALYSIS 


configuration The A-plane is adjusted slightly in Figure 7 The H-pIane 
is given two corrections, as shown m Figures 7 and 9 The C-plane needs no 
further adjustment These adjustments are shown m the matnx 8 U of 
Table 6 The computations of Table 6 proceed in the same way as foi the 
pievious rotations, The oblique factor matnx U, is computed as shown in 

When the factor matrix V, of Table 7 has been computed, the three pans 
of columns are plotted as shown in Figures 10,11, and IS Inspection of 



Figure 9 


these three plots reveals the simple structure which became evident even 
after the first rotation, Since no further rotations are indicated in Figures 
10-12 , they are accepted as the configuration for interpretation. This phase 
of factor analysis cannot be illustrated in the present problem, which is 
fictitious. The reference vectors are denoted A,„ as shown in the columns 
of the oblique factor matrix V 4 The cosines between these leference vec¬ 
tors are shown m the matrix C\ of Table 6. It is seen that these cosines are 
negative, from which we infer that the angles Separating the reference 
vectois are obtuse Hence the angular separations of the corresponding 
primary factors T p can be expected to be acute. The interpretation is that 
the primary factors are positively coi related 
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The correlations between the primary factors T v can be determined by 
the relation 

(29) Rpa = D(A'A.)- l D , 

where the diagonal matrix D is so chosen that the diagonal elements of R pq 
are unity The result is shown m Table 8 The procedures here described 
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Figure 12 


are applicable for rotation of axes in any number of dimensions The in¬ 
vestigator must make his own decision as to whether the axes are to be le- 
stricted by being orthogonal oi are to be allowed to follow the configuration, 
as in the present example He must also make his own decision as to whether 
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to restrict the factors to the positive manifold. It is probably best to impose 
none of these restrictions and to allow the reference vectors to be oblique 
if the configuration demands it or to allow each factor to be either positive or 
bipolar. 

There are several ways in which the results of a rotation can be antici¬ 
pated These become apparent with practice Several have been described 
here It is useful to be able to predict whether a rotation will increase or 
decrease the cosine of the angle between two reference axes. This can be 
illustrated by Figure 4 The cosine of the angle between the given vectors 
A a and A c , denoted A x and B i, is — .266. The negative sign means that they 
are separated by an obtuse angle. If the vectors should be orthogonal, then 
it is clear that A i must move toward B u or else B\ must move toward A j. 
In this diagram, B x is moved toward A 1 to the position B' 2 Hence we can 
expect that the cosine will be increased toward zero unless the vector A \ is 
also moved either by this diagram or by some other diagram If the cosine 
of the angle between two reference vectors were positive, then they would 
have to separate in order to make them more nearly orthogonal When we 
are dealing with cosines of large absolute value, these rules are useful in 
avoiding a rotation by which two reference vectors might be made nearly 
collinear, in which case one of the dimensions would be lost Special cases 
sometimes appear in large studies m which these principles are adapted in 
different ways, but these become evident with practice It is unfortunate 
that the rotational problem cannot be resolved in some routine which can be 
applied without judgment The reason is that m dealing with actual fallible 
data we frequently meet with an incomplete simple structure which is over¬ 
determined for some hyperplanes and not for others Furthermore, we deal 
with some positive or bipolar factors that are clear and overdetermmed, 
some factors that are only suggested by an almost indeterminate configura¬ 
tion in some dimensions, and with the unique variance and error variance, 
which obscure the less clearly defined hyperplanes These conditions are de¬ 
termined by the test batteries that are assembled for investigation and 
which call for new factorial experiments Factorial method as such does 
not give a determinate solution for an ill-conceived factorial experiment 
This is, in one sense, fortunate because it becomes a question of experimen¬ 
tal fact whether a hypothesis represented m the test battery is sustained m 
the factorial analysis A principal-axis solution is always available, and it is 
unique for any arbitrary test battery, but it does not, except in special cases, 
enable us to ascertain whether a psychological hypothesis about factorial 
composition is sustained or refuted 

A single-plane method of rotation 

The rotation problem is, for most computers, one of the most difficult m 
factor analysis. The reason is in the circumstance that most of the rotational 
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methods require some experience before the graphical representations can 
be handled effectively It is our purpose in devising the present method to 
provide a procedure which can be handled by a clerk for most of the work 
The method will be referred to as a single-plane method of rotation. The 
method seems to require little more experience than is necessary to estimate 
the slope of a line through the origin for a set of points Questions of inde¬ 
pendence of rotation are minimized in importance, and the method seems 
to be convergent, so that a hyperplane can be located without a discourag¬ 
ing amount of labor The computer need not think about the summation of 
vectors or about the distinction between a reference vector and its trace in 
the diagram. The method does not postulate a positive manifold, so that 
it can be used as well for locating positive and bipolar factors. The method 
proceeds by locating one hyperplane at a time, so that the investigator can 
begin the more interesting job of interpretation of each factor while the 
remaining ones are being computed. This method has only recently been 
devised, and it has been tried on several sets of experimental data with 
promising results 

While it is not necessary for a clerk-computer to understand the theory 
of the method, it will be given first and then the routine which can be ex¬ 
plained to a novice in computing 

In Figure 13 let Q represent a trial unit reference vector whose direction 
cosines are denoted X ma . The fixed orthogonal reference axes are denoted 
by the subscript m, as in previous factorial work We shall denote the given 
trial reference vector Q and the adjusted trial reference vector P This will 
be done for each trial, so that the reference vector P that is obtained in one 
trial becomes the given trial vector Q for the next adjustment This process 
continues for, perhaps, five or six adjustments, until the reference vector 
seems to be located If there is no simple structure or if the particular hy¬ 
perplane is not defined by the configuration, that fact becomes evident 

It is first assumed in Figure 13 that the trial vector Q is orthogonal to 
one of the orthogonal reference axes M as shown, but we shall see later that 
this restriction can be overcome The projections of the test vectors J on 
the trial vector Q are computed and denoted v ]q (not shown on diagrams). 
We then have 

(lib) Hjq " djrnhmq ; 

m 

where a jm are the direction numbers of the test vector J with respect to the 
fixed orthogonal frame M. 

Since M is one of the unit vectors of the fixed orthogonal frame, we al¬ 
ready have the projections of the tests on this vector They are the values 
a, m in a column of the given factor matrix F If M m the figure represents 
the first centroid axis, then the projections {JMf) are given in the fiist 
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column of F In the figure we have represented, by a set of points, the 
paired values from a column of F and the column of v :q The object is to 
draw a line through the origin with a slope so chosen that a large number of 
the points will he close to the hue This has been illustrated m Figure IS, 
and the slope Si is noted It is simply the ordinate on the line TT for 
a= +1 

Let L be a vector so defined that (1) it has unit projection on Q and (2) 
it is orthogonal to the line TT. The vector L can be expressed 


(31) L = Q - sM 

Let l m3 be the direction numbers of L Then we have 

(32) Imq X m g — S , 

since the vector M is a unit vector on the co-ordinate axis M The new 
vector A p will be eollinear with L and of unit length. 

Let us now consider the oblique case, which is illustrated in Figure 14 
The trial vector Q has the projection \ mq on the vector M, as shown in the 
figure. The projection \ mq then is one of the direction cosines of Q, and M 
is m one of the fixed orthogonal axes When the values of v ]q and a, m are 
plotted on orthogonal cross-section paper, the diagram will look like Fig¬ 
ure 13 The vector (-sill) is shown m Figure l/ h where it is, of course, 
parallel to M. Here we wi ite 

(33) U — Q — sM, 


where U satisfies only one of the requirements for L The vector L is or¬ 
thogonal to TT, and so is the vector U. But the vector L is also defined as 
having umt projection on Q The vector U can be extended by a stretch¬ 
ing factor k, so that it has unit projection on Q Then we have 

(34) L — kU , 

as shown in Figure 14 In order to determine the stretching factor for U to 
give it umt projection on Q, we write 


(35) 

so that 

(36) 


k(UQ) = 1, 

k — ^ 

(UQ) ’ 


where UQ is a scalar product From (33) and (36) we have 


7c (Q - sM)Q 


(37) 
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or 

(38) k = Q 2 - sMQ ' 


and, since Q is a unit vector, we have 
(39) k = 1 - sMQ ‘ 


But the scalar product MQ is merely the direction cosine X m9 for the axis M 
Hence we write 


(40) 


fe = 


1 


sX„ 


The direction number of Q for the axis M is as shown in the figure 
Since the vector (— sM) is parallel to M, the dn ection number of the vector 
U for the axis M is 


(41) 


Xtflg 


The direction number of the vector L for the same axis is l mq) and it is 
(42) Imq = , 


so that the desired direction number can be written m the form 


(43) 


Xjnfl _S 

1 sX m9 


which is a simple computing formula Here the X„ lf /s are the direction cosines 
of the given unit trial vector Q The slope s is read directly from a graph. 
If there are r columns of the factor matrix F, then there are as many graphs 
to be plotted. Each one gives a slope value s m , which is set equal to zero if 
no rotation is indicated 

When formula (43) has been evaluated for each of the orthogonal axes 
M, the new vector L is determined. It is then normalized and becomes the 
new trial vector P 

A numerical example 

A three-dimensional example will be used to illustrate numerically this 
method of rotation Table 9 shows a factor matrix F for twenty variables 
and three orthogonal factors. This table is the same as part of Table 3 of 
chapter vii, except that it is here recorded to only two decimals The prob¬ 
lem is to locate the reference axes A p , which define a simple structure 
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The procedure starts with the selection of a test vector In the present 
example, test 15 was arbitrarily chosen as a starting vector Its dneetion 
numbers are shown m row 15 of F m Table 9. These three direction num¬ 
bers, a, m , are recorded m the first column of Table 10, trial 1 This vector 
is normalized, and it then becomes the first unit vector A, with direction 
cosines X,,,,, as shown m the second column The computations for nor¬ 
malizing the given test vector are shown, in part, m Table 10. We then have 

Table 9 
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2a 2 = 9173, as shown, and l/v / Sa 2 = c, which is the multiplying factor by 
which 

CGLjrn “ 'Ll; Q , 

where c = 1 0441 

The projections v lt are then computed and recorded as shown under trial 1 
in Table 9 The column for the first trial is plotted against each of the 
three columns of the factor matrix F, and the three resulting diagrams aie 
shown m the top row of Figure 15 On each diagram a line is drawn through 
the origin so that it goes through groups of points wherever possible Large 
angular deviations of 30 o -45° from the horizontal are not made unless the 
configuration of points clearly demands so large an angular deviation from 
the A-axis The slope s m of each line is noted, and it is recorded in the next 
column of Table 10 The slope of each line is read graphically It is read 
directly from a vertical line through the point -pi on the base line The 
rest of the calculations for trial 1 are self-explanatory, as shown m Table 10 
The column l mq is normalized and then becomes the column \ mp . The cal¬ 
culations show 2 1 2 = 1 2122, which gives the multiplying factor c — 90827 
The column X m „ shows the direction cosines of A„, which is the trial vector 
A„ for the next trial 

When the direction cosines \ mp of the next trial vector have been deter¬ 
mined, it is useful to compute the scalar product of the given vector Q and 
the new unit vector P. This is the cosme of the angular displacement <f>, 
represented by the trial m question. The corresponding value of sine <j> is 
recorded as shown. For the first trial, sin 0= 28 It will be found that 
trials should be continued until sm <£ is about 10 or less In the present 
example it is found that the second trial gives sin <£=11, which is small 
enough that two adjustments are considered sufficient in this ease 

In the second section of Table 10 these direction cosines are copied m 
the column X ma , as shown The projections v JQ are computed and recorded 
in Table 9 in the column for tiial 2 This column is plotted against the 
columns of the factor matrix F, and we then have the three diagrams m the 
bottom row of Figure 15. Here the groups of points are closer to the hori¬ 
zontal line m all diagrams, a fact which shows that the desired solution is 
being approached A line through the origin is drawn on each of these 
three diagrams which goes through the concentrations of points The slopes 
s m are lecorded in Table 10, and the rest of the computations proceed as 
before 

The value of sm <£= .11, which is small enough so that further rotation 
is not indicated. The projections v,„ of the test vectors on the reference vec¬ 
tor determined in two trials are computed and recorded m the last column 
of Table 9 This last column is, in fact, one column of the oblique factor 
matrix V. 




Figure 15 
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In locating the next reference vector we start with one of the test vectors 
which he in or near the plane already determined We choose, therefore, 
one of the vectors that have small values in the last column of Table 9 We 
might start with any one of the vectors 1, 2, 4, 7, 10, 13, 14, 18, 19 These 
would not all lead to the same plane, but it does not matter in which order 
the planes are determined The starting vector for determining the third 
plane is chosen from those test vectors which have small projections on 
both the first and the second reference vectors A P In this manner one avoids 
starting with a vector that leads to a plane that has already been deter¬ 
mined If the configuration is indeterminate, this fact becomes evident 
when there is no clear indication of where to draw the lines on the graphs, 
or when the adjustments indicate large deviations that would carry the 
new trial vector into one of the reference vectors that have already been 
determined 

It has been found that inexperienced student assistance can be used on 
these computations, most of which can be arranged with check columns 
The check sums have not been shown here for Table 10 It is a fortunate 
circumstance that an error in these calculations is self-correcting, in that 
the adjustment is made by the computations m the next trial It is best to 
provide rather complete check columns so that each step is self-checking 

The work of plotting the diagrams can be considerably reduced by not 
plotting the points for which a is smaller than v These points have raiely 
any effect in determining the slope of the desired line through the origin, 
and they might as well be omitted in plotting the diagrams 

When all the planes have been located by this method, either as planes 
of considerable variance or as residual planes, the columns of the oblique 
factor matrix V should be plotted in pairs Small final adjustments can 
then be made, if necessary, by the methods described earlier in this chapter. 

The present method has been tried on several large factor studies, on 
small test batteries, and on fictitious examples, and it has so far been found 
to be rapidly convergent to the solution, One of its advantages seems to be 
that the computer need not worry about the independence of the several 
angular adjustments in each trial, because each of them affects only one of 
the direction cosines of the new trial vector The difficulty caused by over¬ 
correction does not seem to be a problem with this method. 
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THE METHOD OF EXTENDED VECTORS 


This method has proved to be effective in analyzing a factorial matrix of 
more than three dimensions, but it is applicable only in those cases in which 
a central reference vector can be found on which all the test vectors have 
positive projections The method of extended vectors will be described, first, 
for a simple case of three dimensions, m which the new method gives the 
solution in one diagram if a simple-structure solution exists The method 
will then be described with a five-dimensional fictitious example, which il¬ 
lustrates, perhaps better than the simpler example, just how the method is 
used m practice 

The method of extended vectors for three dimensions 

In the previous chapter we described the method of rotation of axes by 
two-dimensional sections, which are represented on diagrams. Each dia¬ 
gram represented the plane determined by two of the reference vectors, and 
it was plotted from pairs of columns of the factor matrix V In the method 
of extended vectors these sections represent three dimensions. The prin¬ 
ciple of the method will be explained bnefly before the numerical example 
is given. 

Let Figure 1 represent a configuration of test vectors with origin at 0. 
Let the axis 1 represent the first centroid axis, the major principal axis, or 
any other axis that is central in the configuration and on which all the test 
vectors have positive projections. Let the arc CC lepiesent a unit sphere 
with center at 0, and let the reference vector I be of unit length Consider, 
next, the plane PP, which is tangent to the unit sphere and orthogonal to 
the central axis I. In a configuration of r dimensions this plane becomes an 
(r — l)-dimensional hyperplane Let each test vector be extended so that 
its terminus is in the plane PP All the test vectors will then have unit 
projection on the axis I Each test vector is extended by the multiplier 


so that its projection on the I -axis is unity This has been done m Table 1. 
The extended factor loadings were obtained from Table 1 of the previous 
chapter. The multiplier D„ for test j is applied to all the factor loadings a jm 
of that test to determine the corresponding extended factor loadings A jm of 
Table 1 
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In Figure 1 let OAB represent one of the co-ordinate planes containing a 
number of the test vectors. When these have been extended, their termini 
also lie in the tangent plane PP. Since the intersection of two planes is a 
straight line, it is clear that the termini of the test vectors in the co-ordinate 
plane OAB will lie in a straight line in the plane PP and that this linear ar¬ 
rangement of test vector termini should be evident if a plot is made of col¬ 
umns II and III in the extended factor matrix 

The normal A defines the co-ordinate plane OAB of a simple structure. 





The direction cosines of A may be denoted Xi, X 2 , X 3 , and, hence, the equation 
of the plane OAB may be written in normal form as 

( 2 ) XjUi + -f- X 3 X 3 = 0 , 

where sqis the projection of any point (xi, xt, x 3 ) of the plane on the I- axis. 
Since the plane PP is tangent to the unit sphere and orthogonal to the 
/-axis, all points in that plane have unit projection on that axis, so that 
*1 — 1- The equation of the line of intersection AB of the two planes is, 
therefore, 

(®) Xi + \ 3 Xi -f- \ 3 x 3 = 0 , 

when referred to orthogonal axes in the plane PP which are parallel to 
II and III. 
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When a plot is made of columns II and III of Table 1, we have Figure 2, 
where the points are clearly arranged m three intersecting lines The equa¬ 
tions of these lines were determined graphically, and they are as follows ■ 

Trace of X-plane 58 — + 32^3 = 0 , 

Trace of F-plane 55 + 30a:j — x 3 = 0 , 

Trace of F-plane 71 + .83x2 + x 3 = 0 . 

These equations correspond to equation (3), and it will be seen that the con¬ 

stant term of this equation is the coefficient of xi in equation (2) 

Table 1 

Extended Factor Matrix Eo 


Test 


d 6 

I 

II 

III 

i 

1 

6175 

1 000 

-1 117 

209 

2 

1 

3793 

1 000 

248 

- 905 

3 

1 

5038 


808 

752 

4 

1 

1507 


- 241 


5 

1 

1990 


218 

609 

6 

1 

1962 


621 

182 

7 

1 

1682 



- 314 

8 

1 

1792 



377 

9 

1 

1614 

KitTiH 

483 

- 347 

10 

1 

1364 


- 388 

- 402 

11 

1 

1249 

hh 

- 165 

490 

12 

1 

1429 



- 106 

13 

1 

4992 


-1 087 

163 

14 

1 

3947 

1 000 

343 

- 863 

15 

1 

5773 


792 

823 

16 

1 

0684 

1 000 

275 

176 

17 

1 

0352 

1 000 


- 086 

18 

1 

6000 


-1 152 

.266 

19 

1 

4245 


160 

- 926 


1 

5060 


807 

735 


The plot shown in Figure 2 enables us to determine the direction num¬ 
bers of a normal to each co-ordinate plane When these are normalized, we 
have the direction cosines of each reference axis A p 

The successive steps m the numerical work will now be described On 
Figure 2 the three traces are denoted the X-, Y-, and Z-planes, as shown 
The coefficients are recorded m the S-matrix of Table 2 In this case the 
matrices 5 and L are identical The columns of L are normalized as in the 
previous work, and the multipliers d rp are determined Applying these mul¬ 
tipliers to the columns of L, we get the matrix A, the columns of which show 
the direction cosines of the three reference vectors A p It will be seen that 
this matrix A is the same as Aq 4 of Table 6 of chapter x, which was obtained 
after three rotations The slight discrepancies are due to the fact that the 
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two solutions were obtained graphically The columns X, Y, and Z of A m 
Table 2 correspond to the columns B, C, and A, respectively, in the ma¬ 
trix A<m of Table 8 of chapter x The order in which the reference vectors 
are listed in the columns of A, or of the factor matrix, is, of course, imma¬ 
terial 

The oblique factor matrix V of Table 2 was obtained directly from the 
given factor matrix F 0 of Table 1 m chapter x by the transformation A in 
Table 2 The matrix A'A = C shows the cosines of the angles between the 
reference vectors, and the matrix T shows the direction cosines of the pri¬ 
mary vectors Their angular separations are shown in the matrix TT' 


Mr 



Figure 2 

This is the matrix R pqt which gives the correlations between the primary 
factors T p These correlations are the same as the correlations shown in 
Table 8 of chapter x The slight discrepancies are due to giaphical methods 
For example, the correlation + 105 m one table coi responds to the corre¬ 
lation + 112 m the other table 

We have shown here an application of the method of extended vectors to 
a problem m three dimensions. Since this method consists in taking three- 
dimensional sections of an r-dimensional configuration and since r = 3 in 
this case, the whole configuration as to its co-ordinate planes is defined here 
m one diagram, Figure 2 When r exceeds 3, it is necessary to make more 
plots, but not so many as in the method of plotting two-dimensional sec¬ 
tions described m the previous chapter 
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The advantage of the method of extended vectors is that traces can be 
drawn anywhere on the plot, since they need not be radial, as in the two- 
dimensional method A streak of points anywhere on the diagram can be 
brought into a co-ordinate plane The number of rotations is usually much 
smaller than with the two-dimensional method, but the extended form of 
the factor matrix must first be computed The method of extended vectors 
is not applicable when the configuration is of such a character that no cen¬ 
tral vector exists on which all the test vectors have appreciable positive 
projections It is sometimes possible to use the method when all but a few 
of the test vectors have positive projections on some central axis, but special 


Table 2 

Factor Matrix V 


Matrix & —1/ 


X' 

Y' 

r 


X 

Y 

z 


l 

58 

55 

71 

1 

909 

003 

- 003 


ii 

-1 00 

30 

83 

2 

025 

939 

005 

l mT 

hi 

32 

-1 00 

1 00 

3 

007 

023 

957 






4 

476 

727 

OOO 


Si 1 

I 4388 

1 3925 

2 1930 

5 

387 

005 

844 


Vzv 

I 1995 

1 1800 

1 4809 

6 

012 

393 

794 



8337 

8475 

6763 

7 

719 

512 

- 025 






8 

850 

016 

384 






9 

- Oil 

760 

444 






ii 

668 

008 

638 

Matrix A 


X 

Y 

z 

12 

- 007 

610 

628 






13 

956 

034 

— 013 


i! 

483 

466 

479 

14 

- 024 

921 

063 


n | 

- 834 

254 

560 

15 

028 

- 019 

.937 


hi 

267 

- 847 

675 

16 

282 

362 

704 






17 

644 

460 

273 






18 

947 

- 032 

008 






19 

072 

906 

~ 040 

Matrix (7 » A'A 


X 

Y 

z 

20 

004 

032 

948 


X 

1 000 








Y 

- 213 

999 






Cpg 

Z 

- 055 

- 206 

999 






Matrix T* 

X 

Y 

z 

i 

661 

713 

654 

ii 

- 736 

199 

545 

tmp 111 

144 

- 671 

524 


Matrix TT’ 

X 

Y 

z 

X 

1 000 



Rva Y 

229 

1 000 


z 

105 

224 

1 000 
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adjustments aie then necessary to write the final rotated factor matrix for 
the whole battery. 

The method of extended vectors for n dimensions 

The principle that has been described for a three-dimensional example 
will now be applied to a fictitious five-dimensional example We start this 
problem with a simple structure, shown m Table 3. Here we have twenty- 
five variables, and arbitrary factor loadings have been recorded m this fac¬ 
tor matrix for the five columns shown This matrix may be denoted F x 

Table 8 


A Fictitious Simple Configuration F x 



Pi 

P 2 

P S 

Pi 

P t 

h* 

1 


3 

,0 

0 

8 

73 

2 


9 

0 

0 

0 

81 

3 

5 

5 

.0 

5 

0 

75 

4 , 

.5 

7 

.0 

0 

0 

74 

5 

iimm 

3 

3 

0 

7 

07 

6 

.0 

4 

6 

4 

0 

68 

7 

0 

0 

8 

0 


64 

8 

0 

5 

7 

0 

ttl 

mm 

9 


0 

6 

6 

mm 

SiWii 

10 

0 

0 

4 

6 

11 


11 

6 

0 

5 

4 

0 

77 

12 

7 

.3 

3 

0 

0 

67 

13 

0 

0 

0 

0 

.9 

81 

14 

0 

8 

0 

4 

0 

80 

15 

6 

0 

5 

0 

3 

70 

16 

6 

0 

6 

0 

0 

72 

17 

0 

0 

7 

0 

5 

74 

18 

4 

0 

0 

0 

.7 

65 

19 

8 

0 

0 

0 

0 

64 

20 

0 

0 

0 

6 

6 

72 

21 

0 

0 

0 

8 

0 

64 

22 

3 

0 

0 

4 

7 

74 

23 

0 

5 

0 

5 

5 

75 

24 

7 

3 

0 

0 

3 

07 

25 

3 

0 

0 

8 

0 

73 


The communalities are also recorded In Table 4 we have an aibitrary or¬ 
thogonal matrix A*o, which transforms F x into the factor matrix F 0 of Table 6 
We then have the relation 

W F x Aio = To ■ 

We start the problem with this matrix as given, and our problem is to ana¬ 
lyze it m such a way as to rediscover the simple structure of Table 3, which 
will be regai ded as unknown during the computations 

The first step is to extend the test vectors of Table 5 The last column of 
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that table shows the reciprocal of the first-factor loading by equation (1) 
Multiplying each row of F (] by the corresponding value D., we have 


( 5 ) 


DJ 1 ’ o = E o , 


and we then have the extended factor matrix E o of Table 6, m which each 
entry m the first column is unity 

There are four columns that can be plotted to show the configuration, 
and these give six diagrams Only three of these diagrams are shown in 


Table 4 

An Arbitrary Orthogonal Transformation A,o 



I 

II 

III 

IV 

V 

Pi 

447 

- 148 

-.092 

- 832 

- 282 

Pi 

.409 

543 

595 

— 084 

420 

Pi 

447 

- 431 

- 375 

128 

678 

Pi 

.447 

564 

- 554 

307 

- 307 

Pi 

484 

- 438 

438 

438 

- 438 


Table 5 

A Centroid Matrix Fo of the Fictitious Test Battery 



I 

II 

III 

IV 

V 

De 

1 

510 

- 188 

529 

325 

- 224 

1 9608 

2 

368 

489 

536 

- 076 

378 

2 7174 

3 

652 

474 

- 026 

- 304 

- 084 

1 5337 

4 

510 

.306 

370 

- 475 

153 

1 9608 

5 

596 

- 273 

373 

320 

023 

1 6779 

6 

611 

180 

— 209 

166 

452 

1 6367 

7 

358 

- 345 

- 300 

102 

542 

2 7933 

8 

517 

- 030 

035 

048 

685 

1 9342 

9 

536 

074 

- 557 

261 

223 

1 8057 

10 

689 

- 059 

- 263 

454 

- 132 

1 4514 

11 

670 

- 083 

- 464 

- 312 

047 

1 4925 

12 

570 

- 070 

002 

- 569 

132 

1 7544 

13 

436 

- 394 

394 

394 

- 394 

2 2936 

14 

606 

656 

254 

056 

213 

1 9763 

15 

637 

- 436 

- Ill 

- 304 

038 

1 5699 

16 

536 

- 347 

- 280 

- 422 

238 

1 8657 

17 

555 

- 521 

- 044 

309 

256 

1 8018 

18 

518 

- 366 

270 

- 026 

- 419 

1 9305 

19 

358 

- 118 

- 074 

- 666 

- 226 

2 7933 

20 

559 

070 

- 070 

447 

- 447 

1 7889 

21 

358 

443 

— 443 

246 

- 246 

2 7933 

22 

652 

- 129 

057 

180 

- 514 

1 5337 

23 

670 

330 

240 

330 

- 162 

1 4925 

24 

581 

- 072 

246 

- 476 

- 203 

1 7212 

25 

492 

399 

- 471 

- 004 

- 330 

2 0325 
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Figures S, 4> 5, and 6, since these are the only ones that were used for de¬ 
termining the initial locations of the reference vectors. 

The successive computations are shown in the formulae for each part of 
Table 8 and subsequent tables It will be seen that some of the matrices 
have an added column for the first centroid factor, which was used m writing 
the extended factor matrix The new notation is for the matrix M, which is 
identical with A except for the added column for the central axis (I) 1 At 
any stage in the rotations, the matrix A can be written by merely omitting 


Table 6 

Extended Factorial Matrix Eo 



I 

II 

rri 


IV 

V 

1 

1 000 

- 369 

1 037 


637 

- 438 

2 

1 000 

1 329 

1 457 

— 

207 

1 027 

3 

1 000 

727 

- 040 

— 

466 

- 129 

4 

1 ooo 

600 

725 

— 

.931 

300 

5 

1 000 

- 458 

626 


537 

039 

6 

1 000 

295 

- 342 


272 

740 

7 

1 000 

- 964 

- 838 


285 

1 514 

8 

1 000 

- 058 

068 


093 

1 325 

9 

1 000 

138 

-1 039 


487 

416 

10 

1.000 

- 086 

- 382 


659 

- 192 

11 

1 000 

- 124 

- 693 

_ 

466 

070 

12 

1 000 

- 123 

004 

— 

998 

232 

13 

1 000 

- 904 

904 


904 

- 904 

14 

1 000 

1 296 

502 


111 

421 

15 

1 000 

- 684 

- 174 

_ 

477 

060 

16 

1 000 

- 647 

- 522 

— 

787 

444 

17 

1 000 

- 939 

- .079 


557 

461 

18 

1 000 

- 707 

521 

— 

050 

- 809 

19 

1 000 

- 330 

- 207 

-1 860 

- 631 

20 

1 000 

126 

- 125 


800 

- 800 

21 

1 000 

1 237 

-1 237 


687 

- 687 

22 

1 000 

- 198 

087 


276 

- 788 

23 

1 000 

493 

358 


493 

- 242 

24 

1 000 

- .124 

423 

— 

819 

- 349 

25 

1 ooo 

.811 

- 957 


008 

- 671 


the fiist column from the matrix M In the method of extended vectors the 
matrices S, L, H, and C have the additional column for (7). 

The new B -plane 

Figure 3 shows the plot foi columns II and IV of Table 6 A unit vector 
is indicated by an arrowhead on the 77-axis, and another unit vector is 
shown similarly on the 7 V-axis These are the given unit reference vectors 
(77) and (I V) A line has been drawn through a streak of points to leprc- 
sent the trace of one of the planes. It is not necessary that the traces should 

1 The notation “{I)" will be used to denote a unit vector in the axis I. and similarly for 
the othei orthogonal reference axes 
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bound the configuration as in the present ease A normal is drawn through 
the origin, as shown by the dotted line The dnection of the normal is 
shown by the anowhead The direction of the normal is so chosen that the 
projections of most of the points, which represent test vector termini, will 
be positive Since this normal is closer to the II -axis than to the other axis, 
we denote the trace the JB-plane, and the normal is denoted The sub- 



1’iauRB 3 

script of B' 2 represents a new location, and the prime indicates that the new 
vector is a long vector that has not yet been normalized 

Since the new vector is closer to (ZZ) than to (IV), we shall describe it as 
equal to the unit vectoi (ZZ) plus corrections m the directions of (IV) and 
(Z) The vector (ZZ) is then a base vector for B( In Figure S we draw the 
normal B) so that its terminus is directly above the terminus of the unit 
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ector (II). The new long reference vector B( can be described by the 
eetor equation 

6) B' 2 = (II) + d(IV ) + i(I) , 

vhich requires only two numerical values to be found from the graph, 
lamely, d and %. When found, these values are recorded m the B[ column 
>f the matrix Sm in Table 8 


m 



Figure 4 


The value of the coefficient d is read directly from the graph. It is mere¬ 
ly the vertical distance from the terminus of (II) to the terminus of B' 2 on 
F%gure 8, namely, 0 23 These diagrams are ordinarily drawn on cross-sec¬ 
tion paper, where finer gradations are available than on the figures of this 
chapter 

The absolute value of the coefficient i is read directly as the intercept of 
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the tiace on the same axis as the base vector (II) This intercept is 0 79 
The coefficient i is taken as positive if the origin and the new vector are on 
the same side of the trace, as m this case The coefficient i is taken negative 
if the origin and the new vector are on opposite sides of the trace 


r 



Figure 5 

Having read the two numerical values d and i on the graph, we can write 
the vector equation (6) in numerical form, namely, 

(7) Bi = 1 00 (II) + 23 (IV) + 79(1) , 

and these coefficients are written in the column B'„ of the matrix Sm in 
Table 8 

The new C-plane 

In Figure 4 we have the plot for columns II and III of the extended factor 
matrix E a of Table 6 The two unit vectors (II) and (III) are shown by ar- 
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owheads in the II- and 117-axes, respectively A trace has been drawn 
/hrough or near eight points The normal is nearer III than II, and hence 
die trace is labeled the C-planc 

Since the uni t vector (III) is the base vector for this diagiam, we diaw 


v 



Figure 6 

the normal C j so that its terminus has unit projection on (III), as m the 
previous figure The new vector C£ is described by the vector equation 

(8) C' 2 = (III) + 6(77) + t(7) 

The coefficient 6 is lead directly from Figuie J+ as the distance between the 
termini of (777) and C(, namely, — 52 The intercept of the trace on the 
7/7-axis has the absolute value 35, It is taken positive, since the origin 
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and the vector C 2 are on the same side of the trace Hence we have the vec¬ 
tor in numerical form, 

(9) Ci = 1 00 (III) - 52(11) + 35(1) , 

and the coefficients are recorded m the column C' 2 of St>t in Table 8 
The new V-plane 

In Figure 5 we have the plot for columns II and V of Table 6. Here a 
trace has been drawn through, or near, twelve points The normal is drawn 
through the origin to E' 2 so that it has unit projection on ( V ), which is taken 
as a unit base vector The vector equation of E' 2 then is 

(10) K = (V) + KIT) + i(i) , 

where b and 1 are to be determined from the graph The value of b is read 
as — 70 The absolute value of the intercept of the trace on the V-axis 
is .35 It is taken as positive because the vector E( and the origin are on 
the same side of the trace Hence we have the vector equation in numerical 
form, 

(11) Ei = 1 00(7) - .70(11) + 35 , 

the coefficients of which are recorded in column Ei, of S m in Table 8. 

The new A- and D -planes 

Figure 6 shows an example of a graph in which two new planes are de¬ 
termined In locating two planes on the same diagram, one avoids drawing 
traces that are parallel, for reasons to be discussed One trace is drawn 
through seven points Such a configuration is useful m locating the simple 
structure not only because a number of points lie m a straight line but also 
because they extend over some length on the diagram. In the ease of two 
alternative traces with the same number of points, one prefers the one that 
has more length on the diagram 

Another trace is drawn through about thirteen points, and a normal is 
drawn through the origin. Both normals are drawn m the direction of the 
configuration, so that most of the test vectors will have positive projections. 
Since the normal to this trace is nearly parallel to (IV), this trace is denoted 
the D-plane. The first trace might have been denoted the E-plane, but 
such a plane has already been chosen, so this one is denoted the A -plane, 
thus locating five new planes, A to E, inclusive. 

The normal to the D-trace is drawn through the origin and extended so 
that it has unit projection on the base vector (IV) The vector equation for 
the normal then is 

( 12 ) 


D't = -(IV) + e(V) + i(I) , 
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where e and i are to be read graphically. The value of e is — 26, and the 
absolute value of i is 54 This is taken positive because the normal and 
the ongm are on the same side of the trace. We then have the vector equa¬ 
tion in numerical form, 

(13) Di = -100(IT) - 26(F) + 54, 

the coefficients of which are lecorded m columns L>t of &a\ in Table, 8 

The normal to the A-trace is drawn through the origin to the point A t 
with unit projection on (IV), which is arbitrarily chosen as a base vector 
The vector equation for A\ can be written 

(14) A' = (IV) + e(V) + i(I) 

The numerical value of e is —.97, and the absolute value of i is 1.20, 
which is taken positive because A( and the origin are on the same side of 
the trace We then have the vector equation in numerical form, 

(15) At = 1.00(/F) - 97(F) + 1 20(Z) , 

the coefficients of which are recorded in the At column m Soi of Table 8 
It so happened that all the increments i for the vector (I) wei e positive 
in this set of four diagrams In each of the subsequent rotations there will 
be cases of negative increments for (I), as may be seen from the diagrams 
and from the first row of the matrix S in Tables 9, 10, and 11 

Independence of planes 

When the traces have been di awn for the planes, it is desirable to ascer¬ 
tain, before proceeding with the computations, whether the planes are 
linearly independent It sometimes happens that the same plane is repre¬ 
sented by streaks of points on seveial diagrams If these traces were identi¬ 
fied as diffeient planes, one would discover m the subsequent computation 
that the two corresponding normals had an angular separation with a co¬ 
sine of, say, 80 or 90 The result might be that one of the common-factor 
dimensions had been lost. In ordei to avoid the annoyance of the computa¬ 
tional adjustments that would then be necessary, it is well to inspect the 
planes for linear independence This is not necessary m the later rotations 
where the planes are easily identified from one. rotation to the next, but it is 
advisable to make an inspectional check on lineal independence at the start 
of a factorial analysis The check will be described here only for the initial 
location of the refeience vectois 

In Table 7 we have five columns, one for each of the five normals that 
have been chosen In column A there is recorded a plus sign for each test 
variable that has a large positive projection on the normal At, a zero for 
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each test that has nearly vanishing projection on that normal, and a blank 
space for those tests whose projections are neither conspicuously high nor 
vanishingly small In some of the columns there are recorded minus signs 
for those tests which have conspicuously strong negative piojections 

For every pair of columns, theie must be differentiation as to the plus 
signs, the minus signs, and the zeros In the present case we have chosen 
the five normals so that they are linearly independent If we had chosen 
two normals with small angulai separation, they would have had the same 

Table 7 


Projections on New Planes 



A 

B 

c 

D 

E 

1 

4“ 


+ 

0 

0 

2 

0 

+ 
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0 
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4- 
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4 

0 

+ 

+ 

+ 

0 

5 
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+ 
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4- 

0 
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7 

0 

0 

0 

0 

+ 

8 
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0 

+ 
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4* 
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24 
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25 
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" 




tests with high positive projections, the same tests with strong negative 
saturation, and the same tests with small piojections That would indicate 
the fact of linear dependence, and one of the normals could then be discaid- 
ed in favor of a new one which should be linearly independent of the other 
four normals If such a situation arises, it is a useful proceduie to select the 
tests which have no plus signs or minus signs and to find a trace on whose 
normal the missing tests would have significant projection, either positive 
or negative That would, in general, insure the recovciy of all the dimen¬ 
sions of the common-factor space One way to avoid the loss of a dimension 
is to designate a new reference vector by the fixed oithogonal axis to which 
it is the closest, but this procedure is not always feasible 
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Alternative method of writing the S-matrix 

A variant of the method that has been described for Figures 8, 4, 5, and 6 
may be preferred by some students One writes the equation of the trace in 
the form 

(16) «2 Xi + a s x s + d = 0 , 

which might repiesent the A -plane determined fiom a diagram of two col¬ 
umns II and III. If most of the points of the configuration aie on the 
same side of the trace as the origin, then the constant term in (16) should be 
positive If the constant term in (16) is negative, the signs of all the terms 
are reversed If most of the points m the configuration are on the opposite 
side of the trace from the ongm, then the constant term must be negative 
The three coefficients are then recorded in the appropriate column of the 
S-matrix Other variations of these procedures can be easily devised to 
suit individual preferences 

Computations for Table 8 

When the matrix /S 0 i has been written by inspection of the graphs, the 
remaimng computations of Table 8 follow the formulae given and in the 
order given in the table In the initial location of the reference vectors the 
matrices /Soi and L 0 i are identical because the matrix S describes the new 
reference vectois in terms of the fixed oithogonal frame From the matrix 
Loi we obtain the diagonal entries of D, as shown Then we determine H ul , 
which is m this case identical with M oi For subsequent rotations they will 
be different The matrix C i shows the cosines of the angular separations be¬ 
tween the reference vectors These cosines are recorded on the diagrams 
for the next rotation, as shown m all subsequent figures The new oblique 
factorial matrix E i is computed either from H n or from M 0 i In practice 
it is not necessary to write the column I m the extended factor matrices 
The column is here given explicitly to show the complete result of the in¬ 
dicated matrix multiplications The factor matrix E i is used for plotting 
the next set of diagrams 

The first rotation 

The ten pairs of columns of the factor matrix Et are plotted on as many 
diagrams, and these are inspected to select the fii st rotation of the reference 
axes Only three of the ten diagrams were used, and they are represented 
in Figures 7, 8, and 9 Figure 7 shows a type of plot that occurs rather fre¬ 
quently m factor analysis, which indicates an almost rigid rotation of two 
axes, Ai and C i, through a large angle. In all these diagrams we record in 
the corner of the diagram the cosine of the angle separating the two ref¬ 
erence axes that are represented by the plot When these cosines deviate 
markedly from zero, they are used in making a choice of rotation From 
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this diagram we obtain the entiies m columns A 2 and C 2 of the matrix S n 
in Table 9 We have here an example of a negative correction for the vec¬ 
tor (/) in the column C 2 The negative sign of this correction is determined 
by the fact that the vectoi C' 2 and the origin are on opposite sides of the 
trace for the new (7-plane The procedure is the same as that bescnbed for 
the previous diagrams and for Table 8 


c 



Figure 7 

In Figure 8 we have the plot for B and D, on which has been drawn the 
trace for the new L-plane, with a slight adjustment In Figure 9 we have 
the plot for columns C and E, which indicates a rather compelling adjust¬ 
ment in the 17-plane The cosine of the angular separation between E[ and 
Cj is 4- 325 The positive sign of this cosine indicates that the two refer¬ 
ence vectors should separate if they are to be orthogonal or nearly orthogo¬ 
nal In the present case the vector E i moves away from the vector C\ to 
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the new position E« The positive sign of the cosine and the appearance of 
the diagram show that the two vectors E and C will be orthogonal, or 
neaily so In this rotation no adjustment is made in the R-plane This is 
represented by the entry of unity in the B column of Si 2 in Table 9. Hence 
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The second rotation 

Pairs of columns of the oblique extended factor matiix E 2 are plotted on 
ten diagrams, three of which were used to select the adjustments for the sec¬ 
ond rotation of the reference axes. These thiee plots are shown in Fig¬ 
ures 10,11, and 12 

Figure 10 gives two clear tiaces of the planes On the vertical axis we 
see a laige collection of points near the origin and no points elsewhere on 
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the vertical axis Two points, namely, 2 and 4 , are in a radial line, and these 
are combined with the points near the origin for the trace of the new A- 
plane The corresponding entries are made in the matrix of Table 10 
The other trace on this diagram is also clearly indicated There are no 
points on the horizontal axis of this diagram, except those which are at the 
origin, and consequently there is no configuration to hold the reference 


E 



normal B in its given position On the other hand, there are seven points 
in the first quadrant which are radial, and these are used in selecting a newly 
adjusted position for the E-plane In each case the plane is identified by 
adjustments that are less than 45° The corresponding entries are made in 
the matrix £ 23 of Table 10 

In Figure 11 we have a small adjustment m the D-plane as shown, and 
in Figure 12 there is indicated a small adjustment m the E-plane These 
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Computations from Figures 10, 11 , IB 
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diagrams indicate very clearly that the configuration is closing in toward a 
simple structure. No compelling adjustment was found for the C-plane m 
this set of diagrams, and consequently there is only one entry of unity in 
the C-column of matrix Sis, according to which the new reference vector 
C'i is collmear with the given reference vector C\. 


B 



Inspection of the angles m this set of diagrams shows one large cosine, 
namely, AB, which is 4- 580. If the simple structure is orthogonal, or near¬ 
ly so, we should expect the two reference axes, A and B, to separate In¬ 
spection of Figure 10 shows that, when we follow the configuration, the two 
new reference axes, A' 2 and B' 2 , do separate, as shown by their new locations 
in the dotted lines Their given positions are plotted as orthogonal in the 
figure, but their new separation is an obtuse angle Further rotations should 
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give only small angular adjustments The computations for this rotation are 
indicated m Table 10, and the result is the extended factor matrix E a , which 
is used foi plotting the next set of diagrams 

The third rotation 

The third rotation is determined by the traces that are drawn on Figures 
18-17, inclusive. The procedure is the same as for the rotations selected on 
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Figure 11 


the previous diagrams, except for the combination of two rotations for the 
E-plane Figure 17 indicates a small adjustment in the E-plane as shown, 
and Figure 14 indicates a smaller adjustment of the same plane If the cor¬ 
rections are limited to small angular adjustments of a plane with traces through 
the origin, then two or more adjustments can be combined in the same rotation 
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Figures 14 and 17 show the combination of two adjustments of the. E-plane 
in the same rotation In Figure 17 the long vectors that are brought into 
the plane are 20, 21, and 25, and these are near the origin of Figure 14 
Hence a small adjustment in the E-plane on Figure 14 does not affect the 
factor loadings on E of these three tests On the other hand, a small adjust- 



Figuke 12 

ment of the E-plane in Figure 14 involves test 2, which is near the origin 
in Figure 17 The two adjustments are therefore said to be independent If 
tests 20, 21, and 25 had high projections on both A and B, then the two 
adjustments from these diagrams could not be combined because they 
would result m an overcorrection In making the adjustments from Fig¬ 
ures 14 and 17, it would not even be necessary to identify the numbers of 
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the test vectors in both figures because of a principle that is often useful 
When one adjustment moves a configuration slightly out of a plane while the 
other adjustment moves it slightly into the plane, then the two adjustments can 
he combined with assurance that they are independent without identifying the 
test vectors m both diagrams. In the present example the adjustment shown 
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Figure 13 


in Figure 17 tends to move the plane down from the configuration, whereas 
the adjustment shown m Figure 14 tends to move the plane up into the con¬ 
figuration Hence the two adjustments can be combined with assurance 
that they are independent The adjustment in Figure 14 will produce a 
small negative factor loading for test 14, whereas the adjustment m Fig¬ 
ure 17 will produce a small positive factor loading for test 14• The result 
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should be a negligible factor loading for tins test when the two adjustments 
are combined. 

The adjustments selected in Figures 13-17, inclusive, are lecorded m the 
matrix S 3 i of Table 11 The remaining computations for this table proceed 
according to the formulae shown and as described for previous rotations 



Fioure 14 

The result is shown in the oblique extended factor matux 7 ? 4 of Table 11 
The matrix C 4 gives the cosines of the angles between the reference vectors, 
as well as the piojections of these vectors on the unit centroid vectoi (I) 
It is seen from this table that the reference vectors are nearly orthogonal 
In dealing with an oblique factor problem it should be recalled that the 
matrix C gives the cosines of the angles between the reference vectois The 







Table 11 

Computations jrom Figures IS, IJt, 15, 16, 17 
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correlations between the corresponding primary factors are then deter¬ 
mined by the formulae that have been described m chapter x 

In Figures 18 and 19 we have the complete set of ten diagrams which 
show the configuration m relation to a set of reference vectors, and there is 
no further indication of adjustments in the reference vectors This set of 
diagrams is therefore accepted as final for this problem The matrix M 04 of 


c 



Fiourb 15 

Table 11 gives the transformation which carries the given extended factor 
matrix Eq to the final extended factor matrix Ei The same transformation 
carries the given factor matrix Fa, that was obtained by factoring, into a 
factor matrix Vi, which happens to represent a set of orthogonal axes in 
the present problem. We then have 


(17) 


EoMai = Ei , 
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which is ail alternative for 


(18) 


E 3 N 34 — E 4 


We also have 

( 19 ) F oAa4 = Vt. 


Table 12 


Vi = F <Am 



A 

B 

c 

D 

E 

1 

- 016 

319 

803 

- 017 

008 

2 

- 012 

899 

010 

003 

006 

3 

493 

502 

014 

509 

- 015 

4 

- 010 

715 

015 

503 

- 012 

5 

- 010 

315 

697 

- 014 

307 

6 

404 

385 

- 008 

008 

602 

7 

012 

- 005 

- 018 

002 

800 

8 

004 

495 

- 010 

002 

704 

9 

609 

- 019 

- 012 

009 

598 

10 

599 

- 006 

492 

- 002 

401 

11 

406 

005 

- 002 

GOG 

477 

12 

- 001 

320 

007 

702 

277 

13 

- 013 

022 

900 

- 020 

006 

14 

390 

788 

.010 

008 

005 

15 

001 

023 

297 

595 

480 

16 

007 

015 

-.006 

COl 

579 

17 

003 

007 

484 

- 010 

704 

18 

- Oil 

030 

706 

384 

- 009 

19 

- 001 

026 

011 

801 

- 029 

20 

.592 

- 001 

601 

- 005 

002 

21 

801 

- 020 

003 

010 

- 002 

22 

390 

016 

705 

290 

- 007 

23 

486 

499 

507 

- 003 

005 

24 

- 010 

330 

313 

695 

- 021 

25 

800 

- on 

006 

311 

- 013 


This is the formula shown at the top of Table 12, which gives the factor ma¬ 
trix Vi for the twenty-five test vectors with normal length This factor 
matnx is used for purposes of interpretation 

In the piesent fictitious example we are interested to find that we have 
rediscovered the factorial composition of the matrix with which we started 
That factoiual composition is shown m Table 3, and it will be seen that the 
two tables are nearly identical In making the companson, wc must identify 
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the five columns The columns Pi to P 6 of Table S correspond to columns 
D, B, E, A, and C in Table 12 The reason for the arbitrary rearrangement 
of the columns is that m locating the traces of the planes m the diagrams 
and in giving letter identifications to the planes during the rotations we have 
no way of knowing which planes are emerging during the successive rota- 
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Fioore 16 

tions Therefore, m making several factorial analyses of comparable test 
battenes, the order in which the primary factors appear in the successive 
columns of the factor matrix is entirely arbitrary 

When the test vectors are identified by their test names, as well as by 
their code numbers, one learns to identify early in the rotations the well- 
established primary factors In some problems that are concerned mainly 
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with methodology, the planes are given code numbers, which are relied 
upon exclusively for the location of the traces on the diagrams In ex¬ 
ploratory factorial studies it is often found that the primary factors are not 
completely determined in some of the diagrams because the test batteries 
in exploratory work seldom reveal a complete simple structuie It is then 
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Figure 17 


advisable to use all the information available, including the configuration 
as well as the psychological nature of the tests, to determine the co-ordinate 
planes, which are then interpreted as in the nature of hypotheses concern¬ 
ing the underlying parameters or primary factors These must then be in¬ 
vestigated with further factorial studies to determine whether one or more 
hypotheses concerning the postulated functional unities can be sustained. 
The purpose of this chapter is to show that when a clear and complete 
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simple structure exists, as m the present fictitious example, the rotational 
methods that have been described enable us to identify the underlying 
order. The example here used was a five-dimensional orthogonal test bat¬ 
tery The methods of this chapter enable us to handle, m the same manner, 
factorial problems of any number of dimensions in which the underlying 
order is either orthogonal or oblique, 


£ J> 
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CHAPTER XII 

THE SPECIAL CASE OF UNIT RANK 


The one-column factor matrix 

If a correlation matrix can be accounted for by a single common factor, 
then we have a factor matrix of one column, as shown in Table 1 . The fac¬ 
tor loadings can then be denoted with a single subscript, a } for test j By 
the fundamental factor theorem we have FF' = R, so that the diagonal ele¬ 
ments become a] = tij Each eommunality h) is then simply the square of 
the single-factor loading, namely, a* In dealing with actual data it is not 
often that one can account for the correlations by a single common factor, 

Table 1 

Correlation Matrix 



1 

2 

3 

4 

5 


I 

1 

a? 

ns 

ns 

ns 

n 5 

1 


2 


al 

1-23 

ns 

ns 

2 

0-2 

3 

ns 

n a 

a] 

n s 

rs 5 

3 

a 3 

4 

n< 

ns 

ru 

a\ 

7*45 

4 

ai 

5 


ns 

r 3 fi 

Ti 6 


5 

dh 


but this case is of historical interest because it was the case to which stu¬ 
dents of factorial theory devoted almost their entire attention for a quarter 
of a century after Spearman’s first provocative paper on this subject m 
1904 * We shall consider the smgle-eommon-factor problem as a special 
case of the multiple-factor problem instead of by the statistical methods 
formerly used. We shall refer to some of the controversial issues about the 
single-common-factor problem that were debated formerly, and they will 
be discussed here in the light of multiple-factor theory. 

For the case of a single common factor we have 

( 1 ) Tjjc = CLjQ/ic , 

where a, and a*, are the single-common-factor loadings of tests j and k In¬ 
spection of this equation shows that pairs of columns, k and l, must be pro¬ 
portional We then have the theorem that if a correlational matrix is of unit 

* ''Correspondence between General Discrimination and General Intelligence,” Part 
III of “General Intelligence Objectively Determined and Measured,” American Journal 
of Psychology , XV (1904), 268-72 
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rank, then all pairs of columns , or rows , are proportional The converse of 
this theorem is also true, for if all pairs of columns are proportional, then 
all minors of second order or higher vanish, and hence the rank must be less 
than 2 The trivial case of rank 0 is here of no significance The case in 
which the rank is 0 is, of course, identified by the fact that all the mter- 
correlations are 0 That is a case of no scientific interest We have, there- 
foie, the theorem that if all pairs of columns, or rows, of a correlational matrix 
are proportional, then the rank of the matrix is 1 or 0. 

One of the earlier criteria used by Spearman for showing that the mter- 
correlations of a set of tests could he accounted for by a single common fac¬ 
tor was the proportionality of the columns of coefficients Because of sam¬ 
pling errors the proportionality was not exact when experimental data were 
used A later criterion of the degree of correspondence between pairs of 
columns was the correlation between them. In general, if two columns are 
proportional, their correlation is +1 or —1 The degree of proportion¬ 
ality was expressed in terms of the coefficient of correlation. We then have 
the theorem that if the correlational matrix is of rank 1, then the correlation 
between pairs of columns, or rows, is +1 or —1 The converse of this theorem 
is not necessarily true If the correlation between pairs of columns is +1 
or — 1, the columns are not necessarily proportional A specific case which 
disproves the converse is that in which a constant is added to each element 
of a column A linear plot is then obtained which does not pass through the 
origin The correlation is -f 1, but the columns are not proportional 

Spearman’s formei use of the intercolumnar criterion depended on the 
converse, in that the high correlation between columns was the basis for 
the inference that a single factor was sufficient to describe the intercorrela¬ 
tions While the intercolumnar correlation criterion is demonstrably fal¬ 
lible, it would be a rare situation in which a set of mental tests would satisfy 
the criterion of unit correlation, or very high correlation between columns 
when the rank is higher than 1 Hence Spearman’s use of the correlation 
criterion could be defended 

But another type of difficulty appeared with the intercolumnar correla¬ 
tion criterion If all the coefficients lnUaie of the same order of magnitude 
and if these are overlaid with sampling errors, then the dispersion of a col¬ 
umn may be comparable with the sampling errors, and the correlation be¬ 
tween columns may be low because of the restricted range of the entries m 
the coirelation matux The proportionality would still be maintained with¬ 
in sampling errors, but the points in each con elation table would be so re¬ 
stricted in range that the correlation coefficient would not reveal the pro¬ 
portionality The intercolumnar proportionality criterion is therefore su¬ 
perior to the intercolumnar correlation criterion. 

The limiting case of this effect is of some interest If all the coefficients 
in a con elation matrix are equal, then the proportionality criterion is satis- 
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fied, but the correlation coefficient is indeterminate. The proportionality- 
criterion would give the correct inference, namely, that the correlation ma¬ 
trix is of rank 1 We then have 


( 2 ) 

But 


1 


L k Q’jQ'k , 

r k i = a k a t , 
r,i = a 3 a t . 


( 3 ) 


r ]k = r,i - r H , 


and hence 

(4) U ,dk = djO-l — dk&l , 
and 

(5) a, — at, = a t . 

It follows from (1) and (4) that 

(6) a, = a k = = Vrjl 

This limiting case is represented in the theorem that if all the coefficients 
r,it m a correlation matrix are equal, then the matrix is of unit rank, and each 
test has a single-factor loading of Vr^, 

If sampling errors are superimposed on this limiting case, the correlation 
between columns shows only the correlation between random errors This 
correlation should be zero or near zero The mtercolumnar proportionality 
criterion is still valid, and it would be only slightly affected by the sampling 
errors in a finite test battery 

Spearman’s use of the correlational, rather than the proportionality, form 
of the mtercolumnar criterion was determined, probably, by the fact that 
the standard error of a correlation coefficient could be determined, whereas 
the proportionality form of the criterion would require the development of 
an appropriate standard error formula There does not seem to be any fun¬ 
damental difficulty in developing such a formula 

Since the diagonal values of the correlation matrix are frequently dis¬ 
cussed in factor analysis, it is useful to know the theorem that if a correlation 
matrix is of unit rank and if the diagonal elements are unity, then all elements 
m the matrix are +1 or —1. 

It is sometimes useful to note a restriction on the numerical values m a 
single-common-factor correlation matrix. It can be stated in the theorem 
that when the rows and corresponding columns ham been reversed m sign so 
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that all the coefficients m a single-common-factor correlation matrix are positive, 
then the three side correlations of any third-order principal minor are such that 
any one of them is greater than the product of the other two This property fol¬ 
lows directly from the proportionality of the columns and the further re¬ 
striction that the diagonals are positive numbers that do not exceed unity 
We then have 


But r 22 = <1, so that 


and hence 


r 12 _ Tri 
r 13 ~ rj 3 


r 127*23 _ ^ 

r u ’ 


T 121*23 < 7*13 


If a set of three tests have intercorrelations that do not satisfy this restric¬ 
tion, then a diagonal value is greater than unity Such a set of thiee tests 
can be a part of a correlation matrix of rank that is higher than 1 

The number of tests for unit rank 

It is useful to inquire how many factors can be determined from the m- 
tercorrelations of any given number of tests or measures. Here we are con¬ 
cerned about the number of tests which are necessary to determine a single 
common factor In a correlation matrix of n tests we have 


\n(n — 1) 

independently determined correlation coefficients, because these include 
only the side correlations and not the diagonal or self-correlations, which are 
unknown In Table 1 we see that for rank 1 the diagonal elements are simply 
the squares of the factor loadings, which are the unknown communalities 
The corresponding factor matiix has a single column with n factor loadings 
We cannot expect to determine n factor loadings unless we aie given at 
least as many independent coi relation coefficients We then have 

( 7 ) 

or 

(8) n > 3 , 

by which we infer that at least three tests are necessary to determine a single 
common factor In that case we have three unknown saturations a, to be 
deteimined whose squaies are the unknown communalities h* — a^ We have 
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also three intercorrelations, namely, r,k, r j; , r k i. The solution of this case 
can be found from the three intercorrelations, 


(9) 

Tjk G'jQ'k j 

(10) 

TqI = CLjCli j 

and 



( 11 ) 

Divide (10) by (11) Then 
( 12 ) 
so that 
(13) 


rici — (hcQ>i 


Zii = 
ru a k ' 


a k 


a } r k i 

r,i 


Substituting (13) in (9), we have 


(14) 


r,k = 


ajru 
r,i ’ 


from which we have Spearman’s formula* foi the correlation of test j with 
the single common factor, namely, 

<16) - Nil? • 

in which a, is the factor loading of testy. The correlation r, i between test j 
and the single common factor is the factor loading a„ so that r,i = a,i. 

The intercorrelations of three tests can usually be accounted for by one 
common factor, but such is not always the case Consider the two correla- 


Table S 


— 

14 

25 

— 

14 

35 

14 

— 

50 

11 

—■ 

60 

25 

50 

(o) 


35 

50 

(&) 

“ 


tion matrices (a) and (b) in Table 2, which have been used by Wilson and 
Worcester to show that the correlations between three tests cannot always 
be accounted for by one common factor t The correlation matrix (a) is of 

* C Spearman, The Abilities of Man (New York Macmillan Co , 1927), Appendix, 
p xvi, eq (19) 

t E B Wilson and Jane Worcester, "The Resolution of Tests into Two General 
Factors,” Proceedings of the National Academy of Sciences, XXV (January, 1939), 24, n 3 
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rank 1 if we insert the communalities 07, 28, 89, which determine the 
single-common-factor loadings The matrix ( b) is of rank 1 if we insert, the 
diagonals 098, 20, 1.25; but, since one of these is greater than unity, the 
solution is incorrect Matrix ( h ) can be considered as a correlation matrix 
of rank 2, and there will be two general common factors. In this case the 
resolution into factors is, of course, not unique 

The value of a, as determined by equation (15) is subject to fluctuation 
with the sampling errors of the three correlation coefficients in terms of 
which it is expressed It is desirable to minimize this effect by taking an 
average value for a, based on different pairs of tests k and l, with which test 
j is combined. With n tests, the number of ways m which (15) can be writ¬ 
ten for test j is the number of pairs of tests that may be taken from the re¬ 
maining (n— 1) tests, excluding test j Hence the number of ways in which 



(15) may be written is %(n — l)(n— 2). The total number of formulae (15) 
for ascertaining all the single-factor loadings for n tests is ln(n—l)(n~2) 
If we have twenty tests, then formula (15) can be evaluated in 3,420 dif¬ 
ferent ways m determining the correlation of each of the twenty tests with 
the single common factor 

We have seen that three tests are required in order to determine a single 
common factor It is of some interest to see the nature of the indeterminacy 
when only two tests are available Consider Figure 1, in which are shown 
the two test vectors, j and k The correlation between tests j and k can be ex¬ 
pressed as the scalar product of the two test vectors, so that 

(16) La cos , 

but the figure cannot be drawn uniquely unless we know three things, name¬ 
ly, the two vector lengths and their angular separation The same correla¬ 
tion is represented m Figure 1 by the pair of vectors j and k', where the 
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angle and the communahty of k' are both altered so as to retain the same 
given scalar product or correlation The case of one test determines an axis 
that may be defined as collinear with the test itself, but the problem of com¬ 
mon factors does not appear except for two or more tests If equation (16) 
is interpreted for one dimension or common factor, then the cosine of the 
angle is, of course, unity, and we have two communalities which cannot be 
determined by a single correlation coefficient We see, tlieiefore, that two 
tests do not define a single common factor. 

Students of factor analysis who confined their attention primarily to the 
single common factor have used sets of four tests to determine the common 
factor The reason for this is that, m general, three tests determine a single 
common factor uniquely, since there are three intercorrelations to deter¬ 
mine three factor loadings. Hence nothing is proved scientifically by doing 
it A scientific demonstration of any kind must always be overdetermmed 

Table S 



1 

2 

3 

4 


1 

2 

3 

4 

1 

+ 64 

- 56 

+ 48 

- 40 

1 

64 

56 

48 

40 

2 

- 56 

+ 49 

- 42 

+ 35 

2 

56 

49 

42 

35 

3 

+ 48 

- 42 

+ 36 

- 30 

3 

48 

42 

36 

30 

4 

- 40 

+ 35 

- 30 

+ 25 

4 

40 

35 

30 

25 


by the data in order to be convincing Hence the smallest number of tests 
by which a single common factor can be demonstrated is four Of course, 
the larger the number of tests in the correlation matrix, the more convincing 
will be the demonstration of a single common factor, assuming that the 
variables are otherwise properly chosen. 

The case of rank 1 requires that all the correlation coefficients be positive 
or that they can be made positive by reversal of one or more of the meas¬ 
ures. Consider, for example, the correlation matrices in Table 3, which 
are both of rank 1 The absolute values are the same. The first matrix can 
be transformed into the second by reversing the signs of the second and 
then the fourth variables Each reversal represents a reversal of a column 
and then the corresponding row. The reason for this arrangement for,rank 1 
is that the test vectors are collinear and they may be reversed in direction 

The tetrad difference 

Spearman’s later method was to evaluate what are called "tetrad differ¬ 
ences.” The tetrad difference is of the form 


(17) 


T'kmX In TlmTkn — Pi 
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where k and l refer to two rows, while m and n refer to two columns of R 
The four subscripts refer to as many tests, and it is implied that four sepa¬ 
rate tests are involved in the tetrad difference equation Hence the tetrad 
difference is not written so as to include any diagonal terms of R This is 
consistent with the fact that the communalities are unknown. Spearman 
has shown that if only one factor is involved, then all the tetrad differences 
in R vanish 

The tetrad differences have a very simple matrix interpretation They 
are simply the expansions of second-order minors in the correlation matrix 
If the rank of IE is 1, then all second-order minors vamsh The converse is 
also true, for, if the second-order minors vamsh, then the rank must be 1, 
except for the trivial case when all entries are 0 The matrix interpretation 
of Spearman's tetrad difference procedure is that unit rank (i e , a single 
common factor) is established by evaluating separately the second-order 
minors in the correlational matrix 

To establish that a matrix is of any particular rank r, it is, of course, nec¬ 
essary to prove that r is the highest order of the non-vanishing minors. 
Taken literally, this requires that all minors of order higher than r must be 
shown to be 0, but for computational purposes this is probably the most 
awkwaid way possible, especially for the single-common-factor case 

The tetrad difference method of examining a correlation matrix cannot 
be recommended even for the restricted single-common-factor case to which 
it is theoretically applicable The reason is that more effective methods are 
available for ascertaining whether one common factor is sufficient to ac¬ 
count for the mtereorrelations. If more than one factor is required, then the 
tetrad difference criterion is not applicable Because of historical interest, 
some of the properties of the tetrad differences will be described here * 

Because of the great amount of labor that is involved m the computation 
of the tetrads for a large correlation matrix, it is convenient to know how 
many tetrads must be evaluated for n tests in order to cover the whole 
table Since a tetrad difference is the value of a second-order minor which 
does not contain diagonal terms, there are as many tetrads as there are 
second-order minors which do not involve the diagonals Each of these 
minors is defined by two rows and two columns The number of pairs of 
rows that can be taken is the number of combinations of n things taken 
two at a time, or 

(is) ci = . 

* Tetrad differences are no longer in general use The smgle-factor methods have been 
replaced by multiple-factor methods of analysis by most students of factorial theory 
This was not the ease when the matrix interpretation was introduced in 1932 or when the 
first edition of this text was published in 1935 
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The number of possible pairs that can be taken from the remaining col¬ 
umns, since diagonal elements are excluded, then is 


(19) 


rin—i _ ( n 2) (n 3) 

Oo — f;- 


Hence the total number of second-order minors in the correlation matrix, 
excluding diagonals, is 


( 20 ) 


fin fin —2 
2 '“' 2 


n(n — 1 )(n — 2 )(n — 3) 
4 


But, since the correlational matrix is symmetric, it follows that every one 
of these minors is duplicated by a symmetric minor of the same value 
Hence the total number of different tetrads is 


( 21 ) 


1 finrin .-« w(n - 1)(« -2 )(n - 3) 


Since every set of four variables gives three tetrads, it is possible to obtain 
the same result by considering the number of combinations of n things taken 
four at a time Then the number of tetrads is 


( 22 ) 


3C; 


n(n — l)(n — 2 )(n — 3) 
8 


Example: If the number of tests is 20, then the correlation matnx con¬ 
tains 14,535 tetrads 

When the computation of all these tetrads has been made, the result is 
usually that the tetrads do not vanish The inference must then be made 
that one common factor is insufficient to account for the mtercorrelations 
of the tests If it is found that the tetrads do vanish within the sampling 
errors, then the next problem is to ascertain how much of the variance of 
each test is attributable to the single common factor This can be done in 
terms of the correlation coefficients by Spearman’s formula (15) 

Speaiman’s procedure takes into consideration that the tetrad difference 
in (17) does not quite vanish because of sampling errors m the four correla¬ 
tion coefficients If a single common factor is fundamentally present and if 
the four coefficients have known standard errors, an expression for the 
standard error of p can be derived. This has been done by Wishart and by 
Holzmger * The experimentally observed deviations of p from zero should 
not exceed those which might be expected from the standard errors of p 

* John Wishart, “Sampling Errors in the Theory of Two Factors, 1 ' British Journal of 
Psychology, XIX (1928), 180-87; Karl J. Holzmger, Statistical Risumi of the Spearman 
Two-Factor Theory (Chicago. University of Chicago Press, 1930), pp. 6-16 
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This is the central idea in Spearman’s single-common-factor method The 
tetrads in a correlation matrix are first evaluated A frequency distribution 
of these tetrad differences is then made and its standard deviation deter¬ 
mined. If this dispersion is of an order of magnitude comparable with that 
which would be expected from the known standard errors of the tetrad 
differences, then Spearman draws the legitimate conclusion that a single 
common factor is sufficient to account foi the observed intercorrelations 
Applications of formulae of type (IS) give the loading of each test with the 
single common factor whose sufficiency has been established by the fact 
that the tetrads vanish within sampling errors In order to reduce the labor 
of computing probable errors of the tetrads, Spearman and his students have 
developed several abbreviated procedures These are limited, however, to 
the single-common-factor case 

Factorial reduction to unit rank 

There are several types of situations which are of both theoretical and 
practical interest, in which a factor matrix or a correlation matrix is reduced 
to rank 1 Consider, first, the two-column factor matrix in Table 4 If the 

Table 4 


Fader Matnx F Single-Factor Matrix 



I 

II 


I 

1 

60 

30 

1 

6708 

2 

40 

20 

2 

4472 

3 

80 

40 

3 

8944 

4 

20 

10 

4 

2236 

5 

30 

15 

5 

3354 


matrix F is multiplied by its transpose F', it will be seen that the columns 
in B are proportional and that it is of rank 1 The tetrads vanish, and the 
intercorrelations of the tests can be described as well by one factor, as 
shown in the single-factor matrix 

The reason why tins result is obtained is that Matrix F is of unit rank, 
It corresponds to the conceivable psychological situation in which each test 
of a battery calls for two primary mental abilities in the same ratio, al¬ 
though they differ in specificity and reliability In practice, it is possible to 
select from a large table of tests several groups whose intercorrelations are 
high when corrected for uniqueness Each one of these groups of tests can 
be described in terms of one factor, but that factor is not necessarily psy¬ 
chologically significant. The tests may be composites, as illustrated m the 
two matrices One way of avoiding this ambiguity is to work with several 
abilities or factors simultaneously, as is done in the multiple-factor methods 

The simple illustrative case that has been described here can be general- 
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jzed into a useful theorem, namely, that if two columns of any factor 
matrix are 'proportional, then the two columns are of unit rank, and the 
rank of the factor matrix is correspondingly reduced It can be easily seen 
that this situation is more likely to occur m analyzing a small test battery 
Two factors that are actually distinct may then collapse so that they appear 
m the analysis as a single factor, with consequent confusion or error of 
interpretation Only by further investigation of the same domain would 
the error be discovered It is quite likely that this does happen in explora¬ 
tory factoiial investigation where the domain covered by several factors is 
not adequately covered by tests. With these limitations of method the solu¬ 
tion seems to be to explore rather intensively each domain with a large num¬ 
ber of specially constructed measurements until the factorial composition 
of each new test can be anticipated with confidence 

Godfrey Thomson has discussed a very interesting case of reduction in 
rank in his sampling theory. Consider a factor matrix of order nXn t , in 
which each test of unit variance is defined only in terms of the number of its 
elements, namely, n 3 Let there be a total of n t elements involved in the 
domain defined hy the factor matrix Then, in the row j, we have n 3 fac¬ 
tor loadings a, distributed at random, and the remaining factor loadings 
are zeros If each test is of unit variance, we have 

(23) Va) = 1 , 

but, since there are n 3 non-vanishmg elements in row j, we have 

(24) n 3 a] = 1, 


so that 



Let similar notation designate test k Then the correlation r,k will be 
the cross product of the two rows j and k The probability that there will 
be non-vanishmg cell entries m the same column for both rows j and k is 
PiPk, where 

(26) p 3 = ^ and p k = ^ , 

IV l ft l 

so that 



(27) 
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The conelation r ]K then is 


(28) 


r 3 i, = Pjp*,Oja A ri£, 


which reduces to 
(29) 



but this is the well-known case of the correlation expressed in terms of com¬ 
mon elements. It can be expressed as the pioduct of the single factors A } 
and A k , where 

(30) = and = 

so that the conelation matrix is of unit rank This result is approached as 
n t becomes laige Hero we have the case in which a factor matrix with a 
large number of columns n t , constructed m the manner described, ap¬ 
proaches unit rank It is quite another matter to decide whether this type 
of factor matrix is psychologically plausible for the description of mental 
traits and test performances 


Table 5 



1 

2 

3 

4 

5 

1 


56 

48 

40 

32 

2 

56 


42 

35 

28 

3 

48 

42 


30 

24 

4 

40 

35 

30 


20 

5 

32 

28 

24 

20 



1 76 

1 61 

1 44 

1 25 

1 04 


Graphical analysis of tetrads 

If one is interested m the task of tabulating separately the values of all 
the second-order minors in a conelation matrix which Spearman has called 
the tetrads, then some simple graphical methods might be useful Because 
of the proportionality of the columns and rows m this case we see that, if 
the correlational matrix is of rank 1 and if any column k is plotted against 
any other column l, then the plot is linear through the origin with a slope 
which is the ratio of the single-factor loading of test k to that of test l. An 
example can be seen by plotting any pair of columns of the conelation 
matrix of Table 6 

Let two points on the diagtam lepresent tests m and n If the two points 
aie ladial, then 
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so that 

(32) VkmTln VUuTkn = 0 

If the points m and n aie not ladial, the pioportionality of (31) does not 
obtain, and the tetrad (32) does not vanish By this simple graphical meth¬ 
od one can identify the tetrads that do not vanish 



Figure 2 


If the test battery as a whole cannot be descubed by a single common 
factor, the plot will not be linear but will scatter, as shown in Figure 2 
This figure shows the plot of column 9 against column 1 from a correlation 
matrix in a factorial investigation by William Brown and William Stephen¬ 
son * Column 9 represents a test of pattern perception, and column 1 rep¬ 
resents a test of inventive synonyms Although it is evident m Figure 2 that 

* "A Teat of the Theory of Two Factors,” British Journal of Psychology (Gen Sec ), 
XXIII (1935), 352-70 
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a single factoi is not sufficient to account foi the intei conelatio ns and that, 
therefore, all the tetiads do not vanish, it is still possible for smaller groups 
of tests to be of such a character that then intercorrelations can be de¬ 
scribed by a single common factor Their tetrads should then vanish The 
smallest group for which a tetrad can be written is foui tests Two tests 
aie represented by the two columns that are plotted. Any pair of points 
on the diagram determines a tetrad If they lie in a radial line through the 
origin, the corresponding tetrad vanishes If they do not lie in a radial line 
through the origin, the corresponding tetrad does not vanish. 

Figure 2 gives further information about the possibility of a single-factor 
hypothesis for the tests concerned Radial lines can be drawn through 8 
and 17, through 12 and 19, through 6 and 14, with small residuals Any 
one of these three pairs can be combined with tests 1 and 9 to form vanish¬ 
ing tetrads, but it does not follow that all eight of the tests can be included 
in a single-faetoi correlation matux The fact that the points spread on 
different radial lines means that othci tetrads which include tests from dif¬ 
ferent radial lines will not vanish It should be carefully noted that, if the 
tetrads fail to vanish, we can say that one factor is not sufficient to account 
for the correlations, but it still remains a possibility that a Spearman gen¬ 
eral factoi is present in addition to other factors, so that more than one 
factor is required for the battery of tests as a whole We see, therefore, that 
the tetrad criterion is inadequate for disproving a Spearman general factor 
See, for example, Figure 5b and Figure 6 of chapter ix, which illustrate 
three-dimensional configurations that include an orthogonal general factor, 
which we aie here calling a “Spearman geneial factor ” The tetrads do not 
vamsh for such a test battery, but it contains a Spearman general factor 
Neither do the tetrads give any information about the existence of a sec¬ 
ond-order general factor, to be discussed in chapter xvm 

Spearman’s terminology 

Spearman’s hypothesis that the correlations between psychological tests 
can be accounted for by a single general intellective factor has been known 
as the theory of two factors The general factor was denoted g In giving this 
name to his theory, Spearman had m mind two factors, namely, a general 
intellective factor and a specific or unique factor in each test As long as 
factorial investigation was dominated by the central theme of the general 
intellective factor, Spearman’s formulation could be called a theory of two 
factors, one common and one unique With the extension of factor analysis 
to n dimensions and with the principal scientific interest directed to the 
multiple common factois, it is confusing to speak of Spearman’s theory of 
two factors, one common and one unique In order to avoid confusion 
we shall speak of Spearman’s theory of a single common factor, a terminology 
that is consistent with that of multiple-factoi analysis 
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When Spearman found a set of tests for which the intercorrelations wei e 
propoitional, he called the table a hierarchy Such a table had the charac- 
tenstic that the highest correlations could be placed near the upper left 
corner and the lowest conelations in the lower light corner Much of the 
early debate about the question of whether Spearman’s single geneial in¬ 
tellective factor was adequate to account for the observed correlations be¬ 
tween tests turned about the statistical question as to whethei the tables 
of inter correlations exhibited so-called hierarchical order The table of cor¬ 
relations was not treated as a matrix, and the question of its rank was not 
discussed m relation to this problem.* 

Before multiple-factor analysis was intiodueed as an explicit psychologi¬ 
cal method without any assumption that there should be a single dominant 
general intellective factor, there was frequent discussion of other factors 
besides the general intellective factoi g These other factois were regaided 
as secondaiy to the general factor g They were descubed as the “distuib- 
ers” of the tetiad equation, and they were avoided by eliminating similar 
tests fiom the battery For example, if two number Lests were in a battery, 
their correlation would add a number factor to the common-factor vari¬ 
ance, and the tetiad equations were consequently not satisfied Instead of 
adding more number tests to the battery in older to investigate number 
ability, the prefeired procedure was to keep only one of the number tests m 
the battery, eliminating the otheis The number factor was thcieby lele- 
gated to the unique variance, and the tetrads were undisturbed Whenever 
these additional and disturbing factors were investigated, the accepted pro¬ 
cedure was, fust, to extiact a geneial factor which dominated factonal 
studies Then the residuals were investigated in groups for additional 
factors These were always treated as secondary and residual to the domi¬ 
nant general factor g This type of factoi lal analysis is still cuirent among 
British psychologists. Most of them have not yet adopted the multiple- 
factor methods, which include extraction of multiple factors and rotation 
of the reference frame to a simple structure The newer methods leave it as 
a question of fact whether a general factor is in the battery and whether it 
is an orthogonal genei al factor or a seeond-oi dei general factor 

Special factoring methods for unit rank 

Spearman’s single-factor formula 

We shall considei, fiist, Spearman’s formula for factoring a correlation 
matrix of unit lank The single-common-factor loading of each test may be 
determined by a summation piocedure This method is simpler than the 

* L L Thurstone, Theory of Multiple Factoi s (Ann Arbor, Mich , 1933), pp v, vi, 20 
This publication is out of print, but it can be obtained at the University of Chicago 
Libraiy as microfilm document No 1648 
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tetrad method for a single common factor, and it gives more information 
about the variables than the tetrad differences give 

If the correlation r can be described by a single common factor, we have 

(1) f'jk ^ a,Oft 

The sum of column k m the correlation matrix is 


n n 

(33) ^ r,L = 

i=i 1=1 

Since the diagonal terms are unknown, the summation m the left member of 
(33) is unknown and hence not suitable for computing purposes Let (r ) k he 
the sum of column k, omitting the unknown diagonal entry. Then 


(34) 

and hence 

(35) 


n 

Wi = y^rji. - of , 

i = i 

n 

W» = ar^Oi — al 

i-i 


Summing for all given coefficients m R 0 and omitting the unknown diag¬ 
onals, we have 

n n n 

(36) (r) t = ^a k ^a, - ^af, 

k~ 1 j-1 k=l 


or 

(37) 


Cr). — 


2 


Cllc 


v 




where (r)<= (r)i denotes the sum of all the coefficients in U 0 except the 

k 

diagonal ones 

Each of the two terms m the right member of (37) may be expressed in 
terms of summations of known coefficients, as follows 


r,k — a 3 <ik, 


TjJ. 

Oi 


a, 


( 1 ) 

and hence 
(38) 
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Summing for column k, we obtain 




;=i 3 =l 


and, from (34), it follows that 


+ af] = T, 


(41) ^ + 

J=1 jfc=l 

Hence the fust teim of the right member of (37) is 


— + 2(?)r -j- a[ 

T Clu 


The last tcim of (37) may be expressed in toims of summations of coeffi¬ 
cients, as follows 


so that 


and hence 


r,k = a,a k , 


i 2 ,l = ajdf : 


UL = a 2 
<*! ; 


Summing for column k, we have 


n n n 

( 4s ) - 2 X r ' A = = 

S=i j-i r=i 

Summing for column ft, except for the entry in low 1c, we have 


M - 

* j-i i=i 
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Let the sum of the squares of the known coefficients in column k be denoted 
by O’ 2 )*.) so that 

n 

(47) 0’ 2 )x = — r\ k 

3-1 

Then (46) can be written in the form 

(48) + ±«i 

1 k=l 

Substituting (42) and (48) in (37), we obtain 


(49) 


, s 00! , s , 2 O' 2 )* 

0)i = + 2(0- + “2 ~ -^r 


or 

(60) 


/ \ ft, •> _ (0* (^)*) 

(r)t — 2(0 A = 


fiom which it follows that 


(51) 

where 


a _ (r)l — OOi, 

* ( 0 , - 2 ( 0 * 


(r)i — the sum of the coefficients m column k, omitting un¬ 
known diagonal entry , 

(0* = the square of (0* , 

(? 2 ) k = the sum of the squai cs of the known coefficients in col¬ 
umn k , 

(r)< = the sum of all known coefficients in the correlation ma¬ 
trix 


Formula (51) gives the smgle-common-factor loading of each test m a 
correlation matrix of rank one This formula has been given, m different 
notation, by Spearman * 

In Table 5 we have a small correlation matrix of unit rank To find the 
factoi loadings for the first test by Spearman's formula (51) we have 

(r) a = 17600, (r 2 )i = 8064, 

(r)f = 3 0976 , (/■),= 7 1000 , 

which when substituted m Spearman's formula (51) give ai = 80 By the 
same method we get 02 = 70, a 3 = 60, a 4 = 50, and a 6 = 40 
* The Abilities of Man, Appendix, p xvi, eq (21) 
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A logarithmic solution 

Another direct method of solving the single-factor problem is the logarith¬ 
mic solution + The logarithmic formula is 


2 ( w ~~ S log r>k “SS log r > k 

(52) log a k = 2(n - l)(n - 2) ’ 

where j X- k For Spearman’s formula (51) the square of each coirelation 
coefficient must be listed, and for the logarithmic formula (52) the logarithm 
of each coefficient must be listed The logarithmic formula is not applicable 
when some coefficients are near zero, m which case the large negative 
logarithms unduly weight the determination 

A direct summation method 

A thud direct method of determining the factor loadings for rank 1 can be 
derived fiom the giaphical representation of pioportional columns If one 

Table 6 
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5 
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56 

48 

40 

32 
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56 


42 

35 

28 
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48 

42 


30 

24 


4 

40 

35 

30 


20 


5 

32 

28 

24 

20 



Sh 

1 76 

1 61 

1 44 

1 25 

1 04 

7 10 Si 

5k— n i 

1 76 

1 05 

96 

85 

72 

5 34 (si —sO 

si—ru 

1 76 

1 20 

1 28 

1 36 

1 44 

7 04 (n-l)si 

CLk/CLl 

1 000 

875 

750 

625 

500 

3 750 c/a\ 

a\/a\ 

1 000 

7650 

5625 

3906 

2500 

2 9687 t>/of 

(Xk 

80 

70 

60 

50 

40 

c 


column k is plotted against another column l, the slope is the ratio of a A to ai 
For a correlation matrix of ordei nXn, there will be (n — 2) points in such a 
plot because the diagonal entries aie unknown, being the squaies of the 
quantities sought The slope can be determined by the method of aver¬ 
ages, as 


(53) 


^/b &k ? kl 

ai si — r u ' 


wheie Si and Si denote the sums of the known coefficients in columns k and Z 
A numerical example is shown m Table 6 In the first summation low we 
have the sums s A , and the total is s h which here denotes the sum of all 
the known coefficients m the matrix In the next row record values of 
(si — r A i) The sum of this low is ( s t — s i), by which the summations can 
* For pi oof see Thuistone, op cit, p 62 
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be cheeked In the next row record values of (si — ?- w ) The sum of this 
row is (n — l)si, by which that summation is cheeked. In the next row 
record the ratios a k /ai by equation (53). In the next row record the squares 
of these ratios 

The square of the sum of the factor loadings is equal to the sum of all 
the coefficients in the correlation matrix, including the diagonal correla¬ 
tions (chapter vni, equation [9]). We then have 


(54) 


Let the sum of the factor loadings be denoted c and the sum of their squares 
v, which is also the sum of the unknown diagonal coefficients or communali- 
ties We then have 

(55) 

D = s, + v 

and 


(56) 

c = ^ a, , 

so that 

J 

(57) 

s t -f V = c 2 

Divide (57) by al, then 


(58) 

St , v _ e 2 
a\ a\ o] 


Hence 



in which all numerical values aie known except a k Applying this multiplier 
( a i b by equation [51]) to the row m/m, we get the factor loadings ax, 
as shown in the last row of Table 6 

Successive approximations 

In addition to direct methods, there are several methods of successive 
approximation One of these is to insert trial values in the diagonal cells 
and then determine the factor loadings by the centroid method The squares 
of the factor loadings so deteumned are used m the diagonal cells for a new 
trial, and the process is repeated until the factor loadings are found to any 
required degiee of consistency For small correlation matrices the approxi¬ 
mation methods require more trials than for larger correlation matrices 
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For large tables of twenty or more variables, the first trial is usually close 
enough for practical purposes 

When the centroid method is applied to the case of lank 1, theie is no 
pioblem concerning the reflection of test vectors or the changing of signs, 
and the method is then reduced to its first step, namely, the summation 
for each column of the correlation matrix It is assumed here that if there 
are any negative coefficients m the table, the corresponding columns and 
rows have been reveised in sign so as to make all the coefficients positive 
If this cannot be done, then the con elation matrix is certainly not exactly 
of rank 1 The first-factor loadings are then determined by the summa¬ 
tion formula 

( 60 ) aL = v7r 

which is equation (lb) of chapter vm Heie the fnst-factoi loading foi test k 
is deteimined by » which is the sum of column Ic, including the estimated 
diagonal, and r t , which is the sum of all the coefficients m the con elation 
matrix, including the estimated diagonal values This first step of the cen¬ 
troid method is identical with a formula described by Cyril Buit foi a cor- 
lelation hieraichy The pnncipal feature of the centroid method, which 
was devised foi multiple-factor analysis, is the treatment of residuals by 
reflection of test vectors and coiresponding sign changes m older to avoid 
the cncumstance that the column sums of the residuals vanish identically 

In using the ccntioid method for the case of lank 1, we must estimate 
the diagonal values, and consequently the solution depends on taking trial' 
values, which require successive adjustment until the solution is found to 
any lequned degiee of aceuiaoy The adjustments aie small when laigo 
ronelation matrices aic used This proccduie is theoretically mcoircnt, if 
it is the intention to extract fuithei factors fiom the lesiduals In that case 
the fiist-factor determinations again become subject to adjustment with 
new communalities m the diagonal cells 

Hotelling’s iterative method of factoring can be applied to the case of 
unit rank, but it requnes seveial tnals for the successive adjustment of 
the diagonals which become communalities when the conect solution has 
been obtained This solution has the characteristic, when applied succes¬ 
sively to a case of unit rank, that the lesulting factor loadings minimize 
the squaies of the residuals of the side conelations The residuals for the 
diagonal entries are zeio when the collect values have been found 

Limitations of single-factor formulae for multiple-factor analysis 

In dealing with actual data it is always a question of fact whetliei any 
given table of mteicoirelations is of rank 1 When a conelation matrix is 
to be analyzed with the assumption that it is of unit rank, it is, of couise, 
implied that only one factor needs to be extracted The values insetted m 



280 


MULTIPLE-FACTOR ANALYSIS 


the diagonal cells are then the squares of the factor loadings. If the diagonal 
values are known, then the factor loadings are also known In the special 
case of lank 1, it is possible to solve directly foi the diagonal values, which 
are the squares of the desired factor loadings Several such methods have 
been described here, and others could be devised In practice such methods 
are not of great importance because it is rare that a correlation matrix has 
a rank as low as 1. 

The student should be cautioned that if he solves for the first-factor 
loadings by any of the single-factor formulae, and then attempts to extract 


Table 7 
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By equation (51) 
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additional factors m the same manner fiom the residuals, the solution will 
be theoretically incorrect This fact is of considerable theoretical signifi¬ 
cance, and it may be worth while to give a numerical example to show the 
natuie of this error. 

In Table 7 we have a factor matrix for five variables and two factois 
The communahties are listed m the last column of the factor matrix Fig¬ 
ure S shows a plot of the two-dimensional configuration The correspond¬ 
ing correlation matrix is also shown m Table 7 Now suppose that this corre¬ 
lation matrix is analyzed by one of the single-factor formulae We have 
taken Spearman’s formula (51) for this example The first-faotoi loadings 
are shown in the row a*,, and the squares of these loadings are also listed 
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By using these factor loadings we can compute a table of fhst-factoi xesidu- 
a l s as shown at the bottom of Table 7 

The correlation matrix was constructed so that it is of rank 2 After ex¬ 
tracting one factor, the first-factor residuals should be of rank 1, but inspec¬ 
tion of the residuals shows that such is not the case Furthermore, the 
squaie of the first-factor loading of the third vanable is higher than the 
communality of that variable for two factors, as shown m the given factor 
matrix The square of the factor loading is 68, as determined by the single¬ 
factor formula, whereas the communality is only 61 Therefore, there exists 
no axis in the configuration of Figure 3 on which the five tests have the 


£ 



projections given by the single-factor formula These factor loadings are 
theiefoie impossible if the rank is assumed to be higher than one The only 
way in which the single-factor formulae can be justified is to regard them as 
single-factor descriptions of the coi relation matrix, in which case the first- 
factor residuals are regarded as vanable enois If the attempt is made to 
analyze the correlations as due to more than one factor, then the commu- 
nahties must bo adjusted to fit the higher lank Any valid factoring for¬ 
mula should give a set of factor loadings which are the projections of the test 
vectors on an axis in the configuration The same idea can also be expressed 
by saying that the factor loadings should be linear combinations of the col¬ 
umns of correlation coefficients The single-factor formulae give coirect 
factor loadings, or approximations to them, when the correlation matrix is 
regarded as of unit rank or as an appioximation to that rank 



CHAPTER XIII 

THE UNKNOWN COMMONALITIES 

Nature of the problem 

The communahty of a test is its common-factor variance The comple¬ 
ment of this vanance is the uniqueness of the test, as previously defined 
The term “communality”* was adopted to distinguish the common-factor 
vanance from the reliability coefficients, which arc conventionally recorded 
in the diagonal cells of a correlation matrix The notation h'j was adopted 
for the communahty so that h, was the length of the test vector j m the 
geometrical representation of the common-factor space The communahty 
is, then, the square of the length of the test vector, and it is the scalar 
product of test j with itself This intei pretation of the diagonal makes it 
consistent with the side correlations, so that the whole correlation matrix 
repiesents scalar pioducts in the common-factor space. To estimate the 
communalitics is, m fact, to make a cleavage between the common-factor 
variance and the unique variance of the test battery If this is done badly, 
then the identification of the common factors and their relations is ob- 
scuied in the resulting factor matnx 

No matter what values are lecoided in the diagonal cells, we have the 
theorem that the number of linearly independent factors represented by the 
mtercorrelations of n tests is equal to the rank of their correlational matrix R x f 
If unity is written in the diagonal cells of a correlation matnx, we call it the 
“complete correlation matrix Ri,” and its rank is usually equal to its order, 
namely, n If the communalities are written in the diagonal cells, we call it 
the “reduced correlation matrix R,” and the theorem then states that the 
number of linearly independent common factors in a battery of n tests is the 
rank of their reduced correlation matnx R In making a factor analysis one 
usually gambles that the number of common factors is smaller than the 
number of tests, so that r < n. 

Since the diagonals are unknown, we have the problem of determining 
the communalities so that they are consistent with the rank of the side 
coirelations, subject to the restriction that the communalities must be posi¬ 
tive numbers between 0 and +1 If we take the estimated communalities 
too high, then we shall increase the rank of the correlation matrix which is 
due to the diagonals Factoring such a matrix would force some of the 
unique vanance into the factoi matrix, which would give an incorrect re- 

* L L Thurstone, Theory of Multiple Fadoi s (Chicago, 1933), p 8 

1 Ibid , p 20 
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suit If we take the estimated eommunalities too low, then, we destroy the 
Gramian properties of the matrix, so that it is no longer a con elation matrix 
If we take the eommunalities at their correct values, then the lank of the 
side correlations is maintained by the eommunalities, subject to the restric¬ 
tion that they be positive numbers between 0 and +1 

The relations between the experimentally given side correlations and the 
unknown eommunalities can also be stated in the theorem that the smallest 
number of independent factors that will account for the vntercorrelations of n 
tests is the minimum rank of the correlational matrix with the diagonal entries 
treated as unknown positive values between 0 and +1.* This theorem has been 
misinterpreted to mean that the rank is itself a variable, which can be given 
different values, and that the minimum rank should be chosen for which the 
eommunalities take possible values The theorem is operationally true as 
far as computational procedures are concerned, but the essential fact about 
the eommunalities is that they are really determined by the side correla¬ 
tions, which are given The problem is to determine the eommunalities 
which aie implied by the given side correlations When found, they do give 
the minimum rank for which the eommunalities have possible values f 
In dealing with experimentally determined correlation coefficients, it 
must be remembered that the sampling eriors of the coefficients are fortui¬ 
tous and that, consequently, the rank of the con elation matrix with experi¬ 
mental values is, m general, equal to its order, namely, n The scientific 
problem of analyzing a set of experimentally given corielation coefficients is, 
then, to find another correlation matnx of minimum rank whose discrepan¬ 
cies from the experimentally given coefficients are small enough to be lg- 
noied. We may summarize the multiple-factoi problem as follows From 
the experimental data we obtain the side correlations, and the diagonal cells 
are left blank A matrix of minimum rank is to be obtained which deviates 
only slightly from the given side correlations The diagonals are then filled 
in so as to be consistent with the side correlations. We then have the re¬ 
duced correlation matnx which is to be factored In practice the compu¬ 
tational proceduie does not follow this route, which is merely a statement 
of the theoretical problem 

* 1st ed , p 73 

t Professor Godfrey Thomson has discussed this aspect of the factor problem as if the 
object were to minimize the number of common factors (The Factorial Analy^ai of Hu¬ 
man Ability [Boston Houghton Mifflin Co , 1939], chap, vm) Operationally, and as re¬ 
gards computational methods, it is correct to say that when the eommunalities have been 
wntten m the diagonal cells, the rank is a minimum If the eommunalities were taken 
smaller, the rank would rise If the eommunalities were taken larger, then the rank of 
the matnx would also rise This problem is resolved if we realize that the eommunalities 
are actually determined by the given side coirelations, and it is merely a statistical prob¬ 
lem to find their numencal values or approximations to them The theorem is correct 
in that the eommunalities maintain the rank of the side correlations, and this is the mini¬ 
mum possible rank 



284 


MULTIPLE-FACTOR ANALYSIS 


High and low diagonal values 

Among students of factor analysis it is univeisal piactioe to leave the di¬ 
agonal elements as unknown when dealing with the case of unit rank This 
is Spearman’s case of the central intellective factor The tetrad difference 
equation is so written as to avoid the unknown diagonal elements, and 
Spearman’s single-factor formula is also written so as to avoid the unknown 
diagonals When the single-factoi saturations or loadings have been de¬ 
termined, the communalities become the squares of these loadings The cor¬ 
relation matrix is, then, of unit rank, with all cell entries determined Some 
students of the factor problem deviate from this logic as soon as they deal 
with higher ranks Then they insert unity in each diagonal cell, regardless 
of the rank of the side correlations This leads to an erroneous factorial 
solution if the purpose is to analyze the factorial composition of the tests as 
given by the correlation coefficients 

The question as to whether unity or the communalities are to be recorded 
in the diagonal cells is determined by the purpose of the factor analysis If 
the purpose is to reproduce the original test scores as closely as possible by 
means of any specified number of factors, then unity should be written in 
the diagonal cells This is true even for Speai man’s hierarchy if the purpose 
is to reproduce the actual test semes. If the purpose of the factor analysis 
is to analyze the common factors that produced the mtercorrelations, then 
the communalities should be written in the diagonals The objective is then 
to reproduce the mtercorrelations as closely as possible When unity is writ¬ 
ten in the diagonals, the original test scores can be reproduced with small 
residuals When communalities are written in the diagonals, the mtercorre¬ 
lations can be reproduced with small residuals, but the original test scores 
will leave larger residuals because the unique factor m each test is unac¬ 
counted for The problem of analyzing the common factors that are repre¬ 
sented by the intercorrelations is likely to be the more significant for scien¬ 
tific work, but there are legitimate problems that call for unity m the di¬ 
agonals The two objectives—to account for the test scores and to account 
for the mtercorrelations—merge in the same correlation matrix when the 
communalities are unity. They differ even in this case, however, in the ref¬ 
erence frame that is chosen for final description of the variables If the test 
scores are to be reproduced as accurately as possible with a small number 
of factors, the principal axes of the test configuration is the indicated ref¬ 
erence frame If the mtercorrelations are to be reproduced as accurately 
as possible with a small number of factors, then the i ef ei ence frame is chosen 
by the principles of simple structure if it can be found in the configuration 
If it is not found, then the reference axes may be placed accoi ding to clusters 
or in regions of the configuration which can be identified with any con¬ 
venient descriptive names, even though the axes so located do not repre¬ 
sent well-defined functional unities that aie indicated when a simple stiuc- 
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ture can be discovered in the configuration * The principal axes can also 
be used in this case if there is no intention to interpret the factois 

In Table 1 we have a small numerical example that has been prepared to 
show the effect of taking communalities that are too high or too low In the 
first section we have a correlation matrix for three tests Unity has been 


Table 1 


1) d-1 

Correlation Matrix 



Factor Matrix 



1 

2 

3 


I 

II 

III 

1 

1 00 

- 48 

56 

1 

8384 

2797 

- 4677 

2 

- 48 

1 00 

- 42 

2 

- 7809 

0247 

0000 

3 

56 

- 42 

1 00 

3 

8138 

3449 

4677 

2) d = h‘ 









1 

2 

3 


I 



1 

04 

- 48 

56 

1 

80 



2 

- 48 

36 

- 42 

2 

- 60 



3 

56 

- 42 

49 

3 

70 



3) 1 >d>h* 








3 

2 

3 


I 

II 

III 

1 

70 

- 48 

56 

1 

8009 

1240 

- 2078 

2 

- 48 

50 

- 42 

2 

- 6444 

2911 

0000 

3 

56 

- 42 

60 

3 

7273 

1671 

2078 

4) d<h? 


1 

2 

3 


I 

II 

III 

1 

60 

- 48 

56 

. 1 

8031 

1070* 

- 1831* 

2 

- 48 

25 

- 42 

2 

- 5632 

2591* 

0000 

3 

56 

- .42 

40 

3 

6758 

1521* 

1831* 


written in the diagonal, and the corresponding centroid factor matrix is also 
shown Since we have here as many factors as there are tests, the test 
scores should be reproduced exactly, as well as the mtercorrelations In this 
case we have gained nothing by way of condensation or simplification, 
since the number of factors is equal to the number of tests We might, then, 

* An example of a factorial analysis with a set of descriptive reference axes that are 
not primary factors is given m a paper by the writer, "A Multiple Factor Study of Vo¬ 
cational Interests,” Personnel Journal, X (1931), 198-205 The configuration was not 
analyzed for simple structure. 
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just as well use the onginal test scores as they stand, without any factor 
analysis However, if it is desned to lepresent the test scores with fewer 
factois than tests, the factor matnx here shown could be rotated to the prin¬ 
cipal axes, and the major pnneipal axis could be used as the best smgle- 
factoi representation of the scores In that case the intercorrelations would 
not be well repiodueed 

In the second correlation matrix we have the same intercorrelations, but 
we have here the communalities that reduce the matrix to unit rank The 
conesponding factor matrix shows one factor which reproduces the mter- 
correlations exactly The test scores would not be well reproduced by this 
factor matux In fact, no single-column factor matnx can be written that 
would reproduce the test scores, since each test has a unique factor 

In the third correlation matnx we have the same mtei correlations, but 
here we have written diagonal values that are less than unity but gi eater 
than the communalities. The corresponding factoi matrix is also shown, 
and it will be seen that here again it shows throe factors 

Finally, in the fourth correlation matrix we have the same intei correla¬ 
tions, but we have written diagonal values that are less than the commu- 
nalities Here the Gramian piopcrties of the matrix have been destroyed 
The principal minors are no longer positive or zeio, and consequently this 
cannot be a correlation matnx It can be factored by the cenlioid method, 
howevei, and the resulting factor matnx is shown It is seen that the factor 
loadings contain lmagmaries 

The examples of Table 1 seive to illustrate the principle that the commu- 
nalities should be so chosen that they arc consistent with the rank that is 
determined by the intei corielations When the diagonal elements exceed 
the communalities, the rank of the whole matnx is, m general, equal to its 
order The same result happens when the diagonal elements are lowei than 
the communalities, in which case we have lmagmanes m the factor matrix. 
The rank of the matrix does not seem to be m any sense a continuous func¬ 
tion of the communalities On the contrary, the communalities are, in gen¬ 
eral, a singular set of diagonal values that aie deteimmed by the side eone- 
lations In general, the rank of a cori elation matrix is either n 01 r, but 
special eases could be constructed in which diagonal values could be chosen 
to make the rank take intermediate values between r and n This could be 
done in fictitious examples by using the communalities in some of the diag¬ 
onal cells and lnghci values in the lest of the diagonal cells It is evident 
that the diagonal values cannot be chosen so as to reduce the rank below 
that of the given side correlations 

It should bo recalled that the unique variance of each test is determined 
entnely by the othei tests with which it is associated in the test batteiy 
that is being analyzed Except for the vanable errois in the unreliability of 
a test, it can be associated with othei tests that involve the same factors so 
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as to bung its communality up to the lehability of the test This is, m fact, 
the challenge of factor analysis, namely, to formulate hypotheses about the 
nature of the factors that produce the mtei correlations and then to combine 
a test of unknown unique variance with other tests in such a manner that 
the common-factor variance becomes completely known In that case the 
communality is equal to the reliability of the test * 

Minimum rank and minimum trace 

The trace of a correlation matnx is defined as the sum of the eommunah- 
ties, and it may be denoted t Then 


( 1 ) 




The problem has been raised by Godfrey Thomson as to whether the trace 
of a correlation matrix is a minimum when the diagonal elements have been 
so chosen as to minimize the rank The question then is whether or not 
minimum rank and minimum trace coincide m the same set of diagonal 
values The problem was investigated by Walter Ledermann,t who finds 
that, while the attainment of minimum rank by the choice of the proper set 
of diagonal elements gives a small value foi the trace, this is not necessarily 
the minimum possible value for the trace for which the matrix remains 
Giamian. 

For the case of unit rank, the trace of the communalities is a minimum 

* In stating his objections to the use of cominunalitiea, Professor Kelley considers a 
hypothetical battery of ten tests, two of which involve a certain musical capacity not pres¬ 
ent in the other eight tests. The communalities of the music tests include their variance 
In the music factor, since it is common to two tests m the battery Now, if one of the 
musical tests is dropped fi om the battery, the other musical test will have a reduced com- 
munahty The decrement is due to the music factor, which then becomes a specific fac¬ 
tor and is no longer a common factor. ‘‘Capacity in music reflected m part in the one 
teat retained is now specificity, and according to Thurstone not worthy of retention in 
the analytical picture” (Truman L Kelley, Essential Traits of Menial Life [Cambridge 
Harvard University Press, 1935], p 56) It is, of course, not a question of whether the 
specific musical ability is “worthy” of retention The question is how to disentangle the 
music factor fiom other specific factors in the same test and from the error variance in 
that test The separation of the specific music factor from the other specific factors in 
the remaining musical test is not solved by writing unity in the diagonal cells But the 
problem can be solved by using an augmented test battery, writing the communalities 
in the diagonal, and rotating the axes in the common-factor space Professor Kelley 
does not appiove of this procedure, but he has not offeied any other solution for the 
problem 

t "On a Problem concerning Matrices with Variable Diagonal Elements,” Proceeding 
of the Royal Society of Edinburgh, LX, Pait I (1939-40), 1-17 
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except when one of the loadings is laigei than the sum of the otheis If we 
let hi be the largest saturation, we have, for the exceptional case, 


( 2 ) 


n 

hi > ^ h, 

3 = 2 


The exceptional case is obtained by taking another set of values u, so that 
the new diagonal values become 

(3) d, - h) + u, 

A unique set of values of u, is obtained from the relations 

(4) ui = —hi{hi — hi — h s — — A„) , 

(5) Uj ~ hj(h,i ha ha hn ), (j = 2, 3, , n) , 

The trace for the new diagonal values d 3 is then smaller than the tiace for 
the unit rank diagonals h'j by an inclement e, which is 

(0) e= - —(hi — ha -h 3 - — A„) 2 , 


but the rank of the matnx with diagonals d, is (n — 1). The matrix is still 
Gramian 


Table & 


Correlation Matrix Factor Matrix 



1 

2 

3 

4 

5 


I 

1 


2250 

1800 

1350 

0900 

1 

90 

2 

2251 


0500 

0375 

0250 

2 

25 

3 

1800 

0500 


0300 

0200 

3 

20 

4 

1350 

0375 

0300 


0150 

4 

15 

5 

0900 

0250 

0200 

0150 


5 

10 

K 

8100 

0625 

0400 

0225 

0100 




trace 

= 9450 

rank 

= 1 




u, 

- 1800 

0500 

0400 

0300 

0200 



d, 

6300 

1125 0800 0525 

0300 




tiace 

= 9050 

rank 

= (n — 1) 

= 4 




In Table 3 we have a numerical example The coirelation matnx of ordei 
5X5 is of unit lank when the communahties A” are used The eonespond- 
ing single-column factor matrix is also shown in the table The trace for 
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h] is 9450, and the rank is 1 The coirections u, were computed by (4) and 
(5), producing the new diagonals d, as shown The new trace is lower, name¬ 
ly, 9050, and the decrement is ( 20) 2 = 04 by equation (6) with lank 
(n— 1) The same author investigated the minimum trace for higher ranks 
with the same conclusion, namely, that the minimum trace for which the 
matnx remains positive-definite is not necessarily that which gives mini¬ 
mum rank 

On the basis of these findings we can wnte the useful theorems that the 
communalities are a set of diagonal values which retain the rank determined by 
the side correlations The communalities give the matrix its minimum rank, 
namely, the rank of the side correlations The minimum trace does not neces¬ 
sarily give the minimum rank even when the diagonals are so chosen that the 
matrix remains Gramian In piactice, it may be expected that the matrix 
will no longei be positive-definite when the diagonals aie chosen lowei than 
the communalities 

The Heywood case 

A special correlation matrix that has attracted the attention of some stu¬ 
dents of factoi analysis is that in which the lank is 1 (Spearman’s hierarchy) 
but in which one of the diagonal elements must be greater than unity in 
ordei to preserve the low rank A con elation matrix of lank 1 m which a 
diagonal entry exceeds unity is called a “Heywood case ”* This curious 
type of matnx will be discussed here with the example used by Godfrey 
Thomson m his analysis of this interesting case t 

The correlation matrix of Table 3 is of order 5X5, and it is of rank 1, as 
far as this can be determined by the piopoitionality of the columns of side 
correlations If we deteimme the single-factor loadings that will produce 
the correlations, we have the single-column factor matrix as shown m the 
table Since one of these loadings is greater than unity, we have here a Hey¬ 
wood case Instead of woriying about this circumstance as if it were some 
insuperable obstacle m factor analysis, it would seem bettei to regard this 
finding as showing merely that the first test does not conform to unit rank, 
which has been imposed on this correlation matrix If we confine our atten¬ 
tion to the other four tests, we have a clear case of unit rank, but not when 
all five of the tests are considered together 

In the same table we have exhibited one of an infinite number of con¬ 
sistent factor matrices that could be written to cover this case We have 
here a common factor, as shown, with a unique variance for each test 
These are arbitrarily written so that the total variance is unity for each of 
the four tests 2-5, but consistent factor matrices could be written for this 

* H B Heywood, “On Finite Sequences of Real Numbeis,” Proceedings of the Royal 
Society, A, CXXXIV (1931), 486-501 

t Op cit , p 231 
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case in many other ways Test 1 is shown here as a complex test If this 
consistent five-column factor matrix is multiplied by its transpose, then 
by the fundamental factor theorem, FF' — R, the given correlation matrix 
is reproduced The communahty of test 1 is now 9802, which is less than 
unity, and the mystery of the Heywood ease has been lesolved It could 
have been done m many other ways that imply more than one factor m 
the complexity of the offending test 1 It is advisable to restate here the con¬ 
dition under which a conelation matrix can be accounted for by a single com¬ 
mon factor A correlation matrix can be described by a single common factor if 
it is of umti ank (proportional columns ) and if the diagonals are positive numbers 
between 0 and + f If these conditions aie not satisfied, then more than one 

Table 3 


Correlation Matrix Factor Matrix 



1 

2 

3 

4 

5 


I 

1 


945 

840 

735 

630 

1 

1 05 

2 

945 


720 

630 

540 

2 

90 

3 

840 

720 


560 

480 

3 

80 

4 

735 

630 

560 


420 

4 

70 

5 

030 

640 

480 

420 


5 

60 


Consistent Factor Matrix 



I 

II 

III 

IV 

V 

b 

1 

90 

3097 

2000 

1470 

.1125 

9802 

2 

90 

4359 




1 0000 

3 

80 


6000 



1 0000 

4 

70 



7141 


1 0000 

5 

60 




8000 

1 0000 


common factoi is requned to descube the eonelations Mathematically, a 
symmetric squaie matnx with proportional columns and positive diagonals 
can be desenbed as the product of a single-column matrix and its tianspose, 
but the pioduct cannot be a corielation matnx if any of the diagonals are 
negative or if they exceed unity 

The treatment of analogous Heywood cases in multiple-factor analysis 
is similar to that foi unit rank In the case of multiple factors the appeai- 
ance of a diagonal value greater than unity means meiely that the rank of 
the factor matrix is higher than that which produced the Heywood ease * 

* Godfiey Thomson (ibid , p 234) discusses the analysis of Spearman’s hierarchy for 
a large number of tests, with the conclusion that a Heywood case would necessarily arise 
When the number of tests is the same as the number of peisons, the determinant of the 
correlation matnx vanishes Thomson concludes that the determinant of a hieiarchy 
cannot vanish with unity m the diagonals except by introducing a Heywood case, i e , a 
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The number of independent factors 

In the analysis of experimental data m relation to some hypothesis, one 
always consideis, either explicitly or implicitly, whether or not the experi¬ 
mental observations are adequate to sustain the hypothesis Attention usu¬ 
ally centers on the discrepancies between the fallible experimental observa¬ 
tions and the proposed theoretical relations This is the principal domain 
of statistical method But even before such analysis can be undertaken, one 
must satisfy an even more fundamental requirement The number of ex¬ 
perimentally independent observations 01 readings must exceed the num¬ 
ber of independent parameteis m the hypothesis m older for the demonstra¬ 
tion to be deteimmate and convincing This principle is universal, and it is 
so fundamental that every scientist takes it foi gi anted intuitively even 
when he does not formulate it explicitly 

We shall consider a simple example of the punciple Suppose that the hy¬ 
pothesis states that two variables are proportional Wilting this hypothesis 
explicitly, we have y = kx for all observations. There is only one parameter 
in this hypothesis of proportionality The hypothesis would be represented 
by a straight line through the origin, and all observations should he on that 
line Suppose that we take only one observation, which consists of an ob¬ 
served numerical value for x and the associated mimencal value for y 
Plotting the obseivation on cross-section paper, we have only one point 
No one would accept such a demonstiation, because one point on the graph 
does not demonstrate that the two variables are proportional for all pos¬ 
sible values of x and corresponding values of y The hypothesis involves 
only one parameter, and we have only one observation We must have 
more observations than there are independent parameters to be determined 
Two observations which fall on a line through the ongin would look prom¬ 
ising, but they would not be convincing Five or six observations or twenty 
or thirty would be convincing We demand intuitively in any scientific 
demonstration that the number of experimental observations exceed the 
number of independent paiameters m the hypothesis that is to be sustained. 
This is, in one sense, the opposite of the mathematical requirement m solv¬ 
ing simultaneous equations If we have more simultaneous equations than 
unknowns, we are afraid of inconsistencies In scientific work the parame¬ 
ters must be many times overdetermmed, with small experimental incon¬ 
sistencies, before we accept a hypothesis as plausible 

diagonal element greater than unity We have the theorem that if a coirelation matrix 
with unity m the diagonals is of unit rank, then all side correlations are either +1 or — 1 
The determinant then vanishes exactly as is to be expected Then all the tests are per¬ 
fect measures of the single common factor, and there is no ITeywood case If the side 
correlations aie less than unity and if unity is recorded in the diagonal cells, then the de¬ 
terminant still vanishes when n = N, but the rank is then higher than 1 and there is 
no Heywood case 
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In a large and more complex problem one might forget this principle in 
the attempt to determine more parameters than there are experimentally 
independent observations In factor analysis it is useful to know how many 
independent factors can be determined from any experimentally given cor¬ 
relation matrix. In order to find this relation we consider the number of 
experimentally independent correlation coefficients for a battery of n tests 
and the number of linearly independent factor loadings in a factor matrix 
for n tests and r factors. 

The number of mtercorrelations in the correlation matrix of Table 4 is 

n{n — 1) 

2 ’ 

since we regard the diagonal elements as unknown In considering the num¬ 
ber of linearly independent cell entries in the factor matrix, we must take 

Table 4* 




Correlation Matrix 


Factor Matrix 


n teBts 




n teats 


n tests 

r factors 

1 

2 

3 

4 

6 

0 7 8 

i 

ii 

III 

IV 

V 

i 







1 

0 11 

0 

0 

0 

0 

2 

7*12 






2 

an 

022 

0 

0 

0 

3 

7*13 

7*23 





3 

an 

032 

03j 

0 

0 

4 

7*14 

7*24 

7*24 




4 

an 

042 


a « 

0 

5 

7*16 

7*26 

7*36 

7*46 



5 

an 

062 

063 

064 

066 

6 

7*10 

7*26 

7*36 

7*46 

r 6 e 


6 

an 

002 

003 

064 

066 

7 

7*17 

7*27 

7*37 

Ui 

7*67 

T 67 

7 

an 

072 

073 

074 

076 

8 

r 18 

7*28 

7* 38 

7*48 

7*68 

7*08 ?*78 

8 

081 

082 

083 

0H4 

086 



n(n — 1) 
2 

coefficients 

r r(r 

nr -^ 

-lLi 

2 

parameters 


♦This example shows that five common factors cannot be determined from the mtercorrelations of eight teats 


into consideration that the configuiation of test vectors can be so rotated 
that a certain number of zero entries can always be introduced These are 
indicated in the factoi matrix of Table 4 The first axis can be placed so as 
to be colhnear with the first test vectoi. The second orthogonal axis can 
be placed m the plane of the fust two test vectors The third axis can be 
placed orthogonal to the first two test vectors in the space spanned by the 
first three test vectors The fourth axis can be placed orthogonal to the fiist 
three test vectors m the four-dimensional space that is spanned by the first 
four test vectors, and so on This will introduce a number of cell entries 
that are identically zeio The numbei of such entries is |r(r — 1), and hence 
the numbei of linearly independent factor loadings in F is 
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In 01 der that there may be a unique configuration of test vectors for the 
correlation matrix, the number of experimentally independent correlation 
coefficients m R must equal or exceed the number of linearly independent 
parameters in F Hence 


(7) 


r(r — 1) ^ n(n — 1) 
2 ~ 2 


The condition for a maximum value of r foi a given value of n is represented 
by substituting an equality sign for the inequality The condition then 
becomes 


(8) 


r(r — 1) n(n — 1) 
2 2 


or 

(9) 2 nr — r(r — 1) — n(n — 1) = 0 

Solving the quadratic in r, we have the following theoiem In order that 
the correlational matrix R with unknown diagonals for n tests and r common 
factors shall represent a unique configuration, it is necessary that* 


( 10 ) 


(2n + 1) - V8n + 1 
r = ---2- 


The suppression of the positive sign before the radical in (10) is justified by 
the postulate that r < n When the equality sign is used in (10), the value 
of r becomes integral for certain values of n Then the number of independ¬ 
ent parameters of F is exactly equal to the number of expel imentally inde¬ 
pendent coefficients in R. Such is the case when ft = 6 and r = 3 We see 
also, for example, that at least thiee tests are required to determine one 
common factor and that at least four tests are required in order that 
one common factor shall be overdetermined This is the situation m Spear¬ 
man’s tetrad difference equation, which involves four tests for the deter¬ 
mination of a single common factor At least five tests are required to de¬ 
termine two common factors 

Since equation (9) is symmetric m n and r, n can be expiessed explicitly 
in teims of r by analogy from (10), so that 

(11, na +1) +y*±3. 


This relation shoves the minimum number of tests lequired for the determi¬ 
nation of r factors Equation (11) shows, for example, that there must be 
at least eight tests m order to deteimine four common factors. 

* 1st ed , p 76, see also E B Wilson, “Review of Crossroads m the Mind of Man [by 
T L Kelley],” Journal of General Psychology, II (1929), 158 
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An actual count of the correlation coefficients in Table shows 28, wheie~ 
as the count of independent parameters in the factor matrix is 30 Hence 
the five indicated factors cannot be determined fi om the correlations of eight 
tests Four factors can be determined, since 28 > 26, but this calculation 
assumes that the correlation coefficients are infallible With actual data one 
would not ordinarily attempt to find more than two or, at most, three fac¬ 
tors for eight tests 

It is useful to have a table to show the smallest numbei of tests that will 
j ust determine a given number of factors oi the largest number of factors 
that can be determined by a given numbei of tests This information is 
summauzed foi ten factois in Table 5 This table must not be used to de- 


Tableti 


No of 
Factors 

r 

No of 
Testa 

n 

No of 
l^actors 

r 

No of 
Tests 

71 

i 

3* 

6 

10* 

2 

5 

7 

12 

3 

6* 

8 

13 

4 

8 

9 

14 

5 

9 

10 

15* 


* The asterisks rofcr to integral values of 
both r and »■ in (11) 


termine the number of factois to extiact from experimentally given data 
A battery of fifteen tests cannot be expected to determine ten factors with 
any degree of stability In order for a factor analysis to be stable and scien¬ 
tifically significant and convincing, the number of tests must be two oi thiec 
times greater than those which are shown in the table The table shows 
only the number of factois that can be determined fiom the mteicorrela¬ 
tions of a given numbei of tests, as far as the count of independent param¬ 
eters is concerned It does not consider the degiee of ovei determination 
which is necessary for the scientific acceptance of a factorial interpretation 
of fallible data 

Methods of estimating the communalities 

Befoie the numerical work of factoring is begun, it is necessary to esti¬ 
mate the communalities The estimation of communalities will be con¬ 
sidered under thiee general heads, namely, (1) when exact, or nearly exact, 
communalities are desned, (2) when factoring is to be done with adjust¬ 
ment in the diagonal cells after each factor has been extiacted, and (3) 
when factoring is to be done without adjustment in the diagonal cells after 
each factor has been extracted The third case implies any factoring method 
that avoids the computation of tesiduals after each factoi has been com¬ 
puted. 
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The first case is that in which it is desired to estimate the communalities 
very closely Tor large correlation matrices it is seldom necessary to insist 
on close estimates of the- Communalities The reason is that in this case the 
factorial results are not seriously affected by coarse initial estimates for the 
diagonal cells. It is another matter when the correlation matrix is small, 
say, of order 8 or 10 or less For small correlation matrices the factorial re¬ 
sults can be markedly affected by wrong estimates for the communalities 

The best procedure for small correlation matrices is probably to do the 
factoring with such estimates as can be made The resulting factor matrix 
will then give a new set of communalities, which can be reinserted in the 
original correlation matrix The factoring can then be repeated with the 
new eommunahty estimates But in the subsequent tiials one does not 
make any adjustment in the diagonal cells after each factor. After a few 
trials it will be found that the communalities determined by the factor 
matrix are practically the same as those with which the factoring was 
started These values may then be accepted as close estimates of the com- 
munalities It is a fortunate circumstance that the necessity for repeating 
the factoring in older to get stable factorial results and stable communali- 
ties occurs with only small correlation matrices for which the computational 
labor is not prohibitive For large correlation matnces this repetition of the 
factoring is seldom if ever necessary. 

In dealing with experimental data it must be recalled that the rank of the 
cori elation matrix is, in general, equal to its order It is then a question of 
statistical judgment when the residuals are small enough so that they can be 
ignored If the factoring is continued without adjustment in the diagonal 
cells aftei each factor has been determined, the residuals will be identically 
zero after n factors have been extracted This is tiue foi the vanous cen- 
tioid methods and for other methods of factoring. In practice the com¬ 
puter stops the process before such a result has been obtained He stops 
extiactrag further factors when he judges the residuals to be sufficiently 
small At this stage it is evident that each additional factor contributes 
very little to the total variance of each test m the common factors Since 
each factor contributes a small increment to the eommunahty, it is evident 
that the estimated eommunahty must be judged m relation to the number 
of factois that are extracted It follows also that the true communalities 
can be determined only by extracting (n— 1) factois with n tests in the 
battery It would usually be a waste of time to carry the woik to this 
extieme, and consequently the true communalities are not ordinarily de¬ 
termined foi experimental data In determining the best-fitting communali- 
ties by repeating the factoung several times, we must remember that each 
trial should be taken to the same number of factois The levised estimate 
of the communalities should then be stated for the number of factors that 
were determined When it is desirable to obtain a very close estimate of 



296 


MULTIPLE-FACTOR ANALYSIS 


the communalities for a specified number of factois, the ideal pioceduie is: 
to obtain the principal-axes solution to the specified number of factors and to 
repeat this piocess with adjusted communalities, as determined m each trial 
Our second case is that m which a factoring method is used which gives a 
set of residuals after each factor There is then an opportunity aftei each 
factor to adjust the diagonal cells The simplest method is that of recording 
in the diagonals the highest coefficient in the column This method has 
been used in a large number of studies with satisfactory results 

Our third case is that in which the diagonals are not adjusted aftei each 
factor This is necessarily also the ease when the factoring is done with¬ 
out computing residuals after each factor In this ca.se it is necessary to 
make a fairly good estimate of the communalities before the factoring is 
begun If m this method of factoring it should be discovered that a laige 
eri or has been made in estimating the diagonals, it might be necessary to 
repeat the factonng in order to coirect the enor This will be true espe¬ 
cially for small coi relation matrices 

A number of principles and computing formulae will be consideied by 
which the communalities may be estimated Most of these formulae are 
appioximate 

1. Expansion of a minor of order (r+1) 

If the correlation coefficients are infallible and if the rank of the conela- 
tion matiix is assumed to be r, then the communalities can be computed on 

Table 6 


Factor Matrix Correlation Matrix of Hank £ 
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II 
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2 

3 

4 

5 

0 

7 

8 

1 

8 

0 

1 


56 

16 

24 

72 

64 
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24 

2 

7 

4 

2 

56 


38 

49 

67 

72 

63 

53 

3 

2 

6 

3 

16 

38 


48 

24 

40 

52 

54 

4 

3 

7 

4 

24 

49 

48 


34 

52 

64 

65 

5 

9 

1 

5 

72 

67 

24 

34 


76 

52 

35 

6 

8 

4 

6 

64 

72 

40 

52 

76 


68 

56 

7 

5 

7 

7 

40 

63 

52 

64 

.52 

68 


71 

8 

3 

8 

8 

24 

53 

54 

65 

35 

56 

71 





True com- 












munali- 












ties 

64 

65 

40 

58 

82 

80 

74 

73 


the basis of this assumption and tested for consistency In order to com¬ 
pute the communality of a test j, select any minor m the con elation matiix 
which contains the diagonal entry for test j but not other diagonal entries 
and which is of older greater than the assumed rank By the definition of 
rank this minoi must vanish Its expansion is a linear equation in one un¬ 
known, by which the communality may be computed 

In Table 6 are leproduced the intercorrelations of eight hypothetical van- 
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ables. The 1 ank of the matrix is 2 Table 7 shows a minor of order 3 with one 
unknown entry, namely, the communality for variable number 1 In order 
that the expansion of the minor may vanish, the unknown diagonal entiy 
must be 64 If the rank is assumed too high, say 3, it will be found that the 
coefficients of h\, as well as the numerical terms, all vanish This indetermi¬ 
nacy can be removed by assuming a lower rank The fact that the minor of 
Table 7 vanishes does not demonstrate, of course, that the correlation matrix 
is of rank 2. Instead of evaluating all the minors of order 3, one might pro¬ 
ceed to factor the matrix after all the communalities have been determined 
by this method, and it would then be found m the factoring routine whether 


Table 7 



1 

2 

3 

1 

K 

56 

16 

4 

24 

49 

48 

5 

72 

67 

24 


the assumed rank was correct It is evident that this simple method is not 
stable when used on fallible data 

2. Grouping of similar tests 

If a test battery is large and if it contains constellations or clusters of 
similar tests, then the tests m each constellation will be represented by vec¬ 
tors in the common-factor space with relatively small angular separations 
Tfie communality of a test is the square of the length of its vector If the 
angular separations between several test vectors are relatively small, then 
the projection of a test vector on the centroid vector of the constellation 
will be nearly the same as the length of the vector The square of the pro¬ 
jection may be used as an estimate of the communality of the test The 
estimate will usually be slightly too low The projection of each test vector 
on the best-fitting single vector for the constellation can be regarded as an 
approximation to the loading of the test with the single common factor 
which best describes the intercorrelations of the tests in the constellation 

Let the communality to be estimated be denoted h\ and let the tests which 
correlate highest with number 1 be denoted 2, 3, . , s, so that there aie s 

tests in the group The intercorrelations of the tests m this group may be 
assembled in the accompanying tabular arrangement, which is particular- 


12 3 4 


1 

hi 

r 12 

ru 

7*14 

2 

ria 


7*23 

T2 4 

3 

r i3 

(r») 


7*34 

4 

ru 

<Ju) 

(rst) 
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ized for s = 4 It is desired to estimate the communahty h\ from the pi ejec¬ 
tion of test 1 on the centioid axis for this subgroup of test vcctois A slight 
undeiestimation may be expected, because the length of the first test vectoi 
will be slightly longer than its projection on the centioid axis foi the group, 
unless they should happen to coincide To compensate foi this underestima¬ 
tion we may underestimate the communalities of the adjacent tests 2, 3, 4, 
by writing in the three adjacent diagonals the average of each column The 
sum of the adjacent columns 2, 3, . , s, may then be written 

s(2 ir - Sr,) 

(■ - 1 ) ’ 

wlieie 2 r denotes the sum of all known coefficients in the table and Sr, de¬ 
notes the sum of all known coefficients m the hist column 

The sum of all the coefficients in the table can then be estimated as 


(Sr, + h\) -b 


s(2r - Sr,) 
(s - 1) 


and the complete sum foi the fiist column is 

2>i + hf. 


The projection of the first test vector on the centroid axis is the ratio of the 
first-column sum to the square root of the sum of all coefficients in the 
table This projection should be an estimate of hi. Hence we should have, 
as an estimate for hi, 


hi- 


(Sr, + h\y 


S? i -f- h\ + 


s(2r — Sr,) 

(s - 1) 


which simplifies to 

( 12 ) 


(« - l)(ZrQ» 

1 iSr - (2s - l)Sr, 


Equation (12) may be written in several equivalent forms In computa¬ 
tion it may be simplei to determine only two sums, namely, the sum of the 
known coefficients in the first column, 2r,, and the sum of the known adja¬ 
cent coefficients in the half-table which are not m the first column or i ow 
These are marked in parentheses in the present tabular arrangement Let 
their sum be denoted 2 r 0 Then the sum 2r must be equal to 2(2r,4-2r 0 ) 
Substituting this expiession for 2? in (12), we have the equivalent formula 

. (s - D(Sr,) 2 
1 • 2?, + 2s2r 0 ’ 


(12a) 
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whei e Sri is the sum of the known coefficients in the first column and Sro is 
the sum of the adjacent coefficients in the half-table which are marked in 
parentheses in the tabulai arrangement These formulae will be referred 
to as “centroid formulae.” 

For convenience we may list the equivalent formula for small subgroups 
of related tests. When the group contains 3 tests, we have for s = 3, 

(12 b) h\ == Sr 2 ; ^e r23 (3-test formula) . 

When s = 4, we have 

(12c) h\ == (4-test formula) 

It is interesting to note that for a doublet when s = 2 this formula reduces 
to h\ = h\ = rn A doublet is factonally indeterminate, but it will be seen 
that, when the coefficients for a doublet in a large battery do not entirely 
vanish, a graphical method may be applicable. 

3. Grouping of three tests 

A special case of the grouping of similar tests is that of using only tlnee 
tests that are highly correlated in a constellation or cluster. One procedure 
for estimating the communahty of a test j is to select the two tests, k and l, 
which have the highest correlations with j. If a test battery is so construct¬ 
ed that each postulated ability is represented by several tests, it can be ex¬ 
pected that the three tests, j, k, and l, will be represented by test vectors 
with relatively small angular separations The communahty of test j can be 
estimated by one of Spearman’s formulae, namely, 


which is equation (15) of chapter xn It is here written as an approximation 
formula because the communahty of test j m a test batteiy is not necessarily 
determined by only three tests This would be a simple method of estimat¬ 
ing the eommunalities in practical work except for the fact that it is un¬ 
stable because of sampling errors It can be used as one estimate of the 
communahty if it is combined with other more stable estimates It will be 
referred to as a “3-test formula ” 

4. Highest coefficient m each column 

Inspection of equation (13) for estimating the communahty of a test sug¬ 
gests a further simplification in the estimate The numerator contains the 
pioduct of the two highest correlations in the column for test 3 . The de- 
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nominator is the intercoi relation of the two tests so selected, namely, 
k and l If these coefficients are of the same order of magnitude, then the 
estimated communality of test j will be nearly equal to the highest mtei- 
correlation in column j This is the method that has been used in a large 
number of studies in which the factoring was done by one of the centroid 
methods, adjusting the diagonal values of the residuals after each factor 
had been computed This simple method of estimating communalities is 
useful only for large correlation matrices. It is not applicable to small tables 
The principles of No 3 and No 4 can be combined m another computing 
foimula Let test 1 be the test whose communality is to be estimated, and 
let tests 2, 3, and 4 be several other tests which have high correlations with 
test 1 The coefficients may be shown in the accompanying tabular an ange- 



1 

2 

3 

4 


1 

2 

5 

6 

1 

hi 

ria 

7*13 

ru 

1 

72 

56 

72 

64 

2 

n% 

hi 

7*23 

ru 

2 

56 

72 

67 

72 

3 

ri3 

r 23 

w- 

ru 

5 

72 

67 

76 

76 

4 

ru 

4 

TU 

hi 

6 

64 

72 

76 

76 


S 2 64 2 67 2 91 2 88 11 10 


ment for four tests If, as a first estimate for each test, we record the highest 
coefficient m the column foi this subgroup of four tests, we have the numeri¬ 
cal entries which were obtained fiom Table 6 The tests 2, 5, and 6 are 
those which correlate highest with test 1 The first-column sum is, then, 
(Sri + ti), where Sri denotes the sum of the known coefficients m the first 
column and U is the highest coefficient in that column as recorded m the 
diagonal cell The sum of all the coefficients in the table is (Sr+Si), where 
Sr is the sum of all known coefficients By the centroid formula we then 

Sr, + l. 

Vsr + S/' 


have 


(14) 

7ii = 

so that 


(15) 



(Sr, + tjy 
Sr + Sf ' 


For test I we then have the numerical estimate (2 64) 2 /ll 10, which gives 
58 as an estimated communality The true value in this case is 64. Equa¬ 
tion (14) is a centroid formula 

5. Linear dependence of rows and columns 

If the rank of a coi relation matrix is r, then any row may be expressed 
lincai ly m terms of any r independent rows. It may be possible to generalize 
this principle into a method of computing the communalities for fallible 
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data. In using this method one assumes a rank for the correlation matnx, 
and it is also essential that the rows chosen for the computation he linearly 
independent 

The idea of using linear dependence of rows and columns of the correla¬ 
tion matnx could perhaps be used for estimating the communalities if the 
matrix weie divided into four or into nine sections The method can be out¬ 
lined with the adjoining table showing a correlation matrix divided into nine 

jTSii Rn Rib 

Tlv. Rm 22 2 a 

Rb l lib 2 22sa 

square sections Let us assume that the tests have been divided into clus¬ 
ters by inspection of the correlations Each cluster should be represented 
by at least one test in each of the three groups into which the tests have 
been divided The notation m the table means that R23, for example, is 
the square section showing the correlations of the tests m the second group 
with the tests m the third section With this prearrangement of the tests 
in three groups and with the assumption that the rank does not exceed 
one-third of the number of tests, it should follow that the third group of 
columns can be represented by a linear transformation of the second group 
of columns, and similarly for the rows. If T denotes a linear transformation, 
we should write R 3 iT = R 2 i, R 32 T = R 22 , R 33 T — R 23 Since the sections R 3 \ 
and Rm aie known, the transformation T could be determined and hence 
the other sections The side correlations of the symmetric sections so com¬ 
puted should not deviate far from the experimentally given values m the 
three diagonal sections, and the communalities would then be estimated 
This idea has not been developed into a practical computing method foi the 
communalities, but it might be done * 

6. Sectioning of the matrix 

The unknown communalities can be solved for by sectioning the correla¬ 
tion matrix. This method has not been used because it requires moie com¬ 
putation than the simplei methods that have been found adequate m prac¬ 
tice, and it is not a stable method for fallible data The sectioning theorem 
will be given here because of its interest m relation to the correlation matrix 
If any matnx of rank r is sectioned into a composite square matrix of order s, 
where s>i, then the determinant of the composite matrix vanishes. 

The matrix will be said to be sectioned when the columns have been di¬ 
vided into s groups, and when the rows have also been divided into the same 

* Since this chapter was written the idea has been developed mto a computing foi - 
mula, as described in a later section of this chapter on “Further Studies of Linear De¬ 
pendence ” 
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number of groups Let r — 2 as an example Since s>r, we may let s = 3. 
Then the n columns of B will be divided into three groups of p, ( q—p ), and 
( n—q ) columns, respectively, while the n rows of R will be divided into 
three groups of i, ( u — t ), and ( n—u ) rows, respectively The matrix It will 
then be sectioned 

The composite matrix will be defined as the square matrix of order s in 
which the entries are the sums of the elements in the corresponding parts of 
the sectioned matrix The example of Table 8 illustrates the formation of a 


Table 8 


14 

12 

6 

8 

2 

6 

104 

21 

9 

17 

7 

6 

3 

4 

1 

35 

30 

15 

20 

6 


composite matrix This 4 X 5 matrix is of rank 2 It has been sectioned into 
a 3X3 square matrix by arbitrarily dividing the columns into three groups 
of 2, 2, and 1 columns, respectively, and by arbitrarily dividing the rows 
into three groups of 1, 1, and 2 rows, respectively The composite matrix is 
shown m Table .9, Its determinant vanishes 


Table 9 


20 

14 

2 

110 

30 

17 

78 

42 

6 


The proof of the theorem will be written for rank 2, but it can readily be 
generalized for any rank If R is of rank 2, it is possible to find two rows that, 
are linearly independent Let these be the first and second rows Then the 
elements of the jth row can be expiessed as a linear function of the first two 
rows so that 

( r ,i = m x r u + m t r n , 


(16) 


r i2 = mir 12 + , 


r,i = m, x r lk + 
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It is evident that the sum of the first p entries of row j can also be expressed 
as the same linear function of the corresponding sums in the first two rows. 
We then have 

v p P 

(17) 2 r, j. = m, 'y'r lh + m 2 ^r 2 1 

k=i fc=i i=i 


Similar summations may be written for the other two groups of columns so 
that 


(18) 


u « 

^ r, k = mi ^ ru + m 2 ^ r**. , 


fc=o>+n 


fc=(P+l) 


Wf>+1) 


^ ( r,k = mi ^ hk + m 2 ^ 7'm., 


U=(a+i) 


/==(«+!) 


*—(a+i) 


These summations may be. represented m an n-X3 matrix as shown m 
Table 10. Since each of the rows can be expressed as a linear function of the 


Table 10 
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T\k ~ &12 

P + 1 
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^ ^ Vik ~ & 2 i 
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^ ^ ro.k = bus 
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^ ^ T-\k — &31 
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7*3* = &32 

P+1 
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^ ^ Tsk = &33 

«+l 




V 

Tnk ~ &nl 

1 

Q 

Tnk ~ &fi 2 

P+1 

n 

r„j; = b„3 

o+l 


first two rows, it follows that the rank of this nX3 matrix is also 2 The col¬ 
umns may be so arranged that the third column of this matrix may be ex¬ 
pressed m terms of the first two columns This reduction by columns is 
similar to the reduction by rows that has been described This reduction 
by columns gives a 3X3 composite matrix whose rank is 2, and hence its de¬ 
terminant vanishes If the rank of R is equal to or greater than the order s 
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of the square composite matrix, then the determinant of the composite does 
not necessarily vanish 

The communahties can be solved for by sectioning, if this is done m such 
a manner that one row and the corresponding column are taken as sections 
Then intersection will be a section with a single entry, namely, the commu- 
nality to be detei mined 

7. Expansion of principal minors of order (n — 1) 

It is possible to write n principal minors of order (re —1) m a square 
matrix of order n If the expansion of each of these n principal minors of 
order (n— 1) is set equal to zero, the rank of the matrix is assumed to be not 
greater than (n— 2) This follows from the property of a Gramian matrix 
that if all of its principal minors of order m vanish, then the rank of the 
matrix does not exceed (m — 1) Since there are n principal minors of order 
(n — 1), their expansions give as many equations as there are unknown diag¬ 
onal entries A unique solution is obtained if the inequality (10) is satisfied 
If this inequality is not satisfied, there should be no unique solution In this 
method it is not necessary to know the rank. These considerations are of 
some analytical interest, but they do not seem to lend themselves to com¬ 
puting purposes. 

8. Expansion of principal minors of order (r+a) 

This should be a special case of the preceding method but less laborious 
It is not necessary that the rank be known, but it is assumed that (r+a) is 
taken larger than the rank The simplest case is that in which a = l This 
method requires that the number of tests covered by the expanded principal 
minors be such as to satisfy inequality (10), even though all the tests in the 
correlational matrix are not utilized. The development of this type of anal¬ 
ysis would be of intereat, but so far it has not led to practical computing 
methods. 

9. A summation formula 

Let the test whose communality is to be determined be denoted number 
1, and let number 2 be the test which correlates highest with test 1 These 
two tests will usually have a small angular separation, and hence their m- 
tercorrelations should be nearly proportional. Hence 

( 19 ) H rii ft? 

Sr 2 — rj 2 ' r l2 ’ 

where Sri and Sr 2 are the sums of the known coefficients m the two columns 
This gives the approximate computing formula 

-L 2 -i. ris(2ri - ru) 

1 • (2i-.-ruT* 


(20) 
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which may be referred to as a “summation formula ” Its source of insta¬ 
bility is the single coefficient ri 2 , whose variable error influences directly the 
estimate of h\ This formula makes the assumption that the two columns 1 
and 2 are of unit rank and hence that they are proportional. 

10. Spearman’s formula 

The summation formula of equation (20) can be generalized for a group 
of foui correlated tests, including test 1, which may be assumed to be nearly 
collmeai From (20) we may wntc for each of the thiee tests 2, 3, 4, 


( 21 ) 


hi 


r t2 (2ri — ria) ri 3 (Sr-i — r 13 ) __ r u (Sri — r u ) 


(2 r 2 — ru) 


(Sr 3 — r iS ) 


(Sr4 — ri ( ) 


where 2?-* is the sum of the known coefficients in column x From these 
three expiessions we write 


( 22 ) 


M = 


r ia(Sr! - r 12 ) + ria(2r l - ?i 3 ) + r 14 (2r! - n<) 
(S? 2 — r i2 ) + (Si 3 — ri 3 ) + (2 r 4 — r u ) 


which simplifies to one of Speaiman’s foimulae for the single common fac¬ 
tor, namely, 


(23) 


__ (2rQ 2 - (Srj) 

1 ' 2r — 22n ’ 


which is the same as equation (51) in chapter tu m different notation 
Here Sr is the sum of all known coefficients for the subgroup and 2 rl is the 
sum of the squares of the coefficients in the first column This formula of 
Spearman should give a good estimate of the communality, although it was 
not originally written in the present context The formula assumes ap¬ 
proximation to unit rank for the subgroups. 

11. Grouping of four tests 

Spearman’s 3-test formula, equation (13), can be adapted to the case of 
foui correlated tests wheie test 1 is the one whose communality is to be 
estimated In the 3-test foim we have 


(24) hi = , 

r 23 

and this can be extended to four tests in the relations 


_1_ rilTu VlsTu _A_ 1 llTli 

r 23 ‘ r n r 2 4 


(25) 
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These three formulae can be combined into a single 4-test formula by talc¬ 
ing the geometric mean of the three expressions, and we then have 


(26) 




' 1-12? 13?’l4 
r 2i r 2 4?’34 


12. Graphical methods of estimation 

If two test vectors (1 and 2) are nearly collmear, the correlations of their 
respective columns should be propoitional to the latio of hi/hi We should 
then have 

r_ 2 j _ hth, cos 

' ' ri, h\h, cos fa,' 

and, since the two cosines should be neaily the same, we have 


(28) 

II 

ht 

hi 

or 



(29) 

«i 

11 

= S 2 p 1 , 


Hence theie should be a linear plot of u 3 against ri, thiough the ongm 
Since the correlation m should be nearly equal to hih 2 , we should have, ap¬ 
proximately, 

(30) h* = — and hi == r l2 s 2 i, 

s 2l 


by which the communalitics can be estimated When the plot is clcaily 
linear through the origin, this method of estimation is good except for the 
variable error in r 12 A clear linear plot indicates that the two test vectors 
are nearly collmear This method of estimation is probably the best 
one m dealing with a doublet, in which case theie is only one appieciablc 
coefficient in the column 

If two tests, 2 and 3, are coplanar with test 1, a computing formula can 
be written for estimating the communalitics by plotting against 1 1 , 

The plot should then be lmeai but not ordinarily through the ongm Fiom 
the slope and one of the intercepts one could then determine the lmeai com¬ 
bination of columns 2 and 3, which lepioduees column 1 This coplanar 
method is of some mteiest, but it is not, m this foim, leadily applicable. 
The collmear foim of graphic estimation is often quite useful 
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Summary of methods 

When a column of the correlation matnx contains a number of coeffi¬ 
cients that are quite high, the estimate can be made with one of the formulae 
which include the conelations of seveial tests In general, we should expect 
instability with fallible data when the enoi in the estimate is dnectly af¬ 
fected by the eiror m a single coefficient as m equations (13), (20), and 
(30) 

When a column contains only one high coefficient, none of the computing 
foimulae apply because we then have essentially a doublet The estimate 
might then be made by the giaphieal method A pure doublet in which 
there is only one non-vanishing coefficient in the column, while all the rest 
aie zeios, is factonally indeterminate Theoretically, this case is the same 
as that of a con elation matnx foi only two vanables There is only one 
conelation which is insufficient to determine two communalities In piac- 
tical work one seldom finds a pure doublet, but one does find approximations 
to them If a doublet is to be retained in a batteiy, one can sometimes make 
a good estimate graphically if the remaining coefficients do not entirely 
vanish Another proceduie is to set both communalities equal, in which case 
they aie both equal to then correlation, the highest in the column 

If the factoring is to be done by a method with adjustment in the diag¬ 
onals after each factor and if the batteiy is large, say, more than twenty oi 
thirty variables, then one may record the highest coefficient of the residual 
column m each diagonal cell. This procedure is not adequate foi small cor¬ 
relation matrices For very small tables of, say, less than ten vanables, one 
may need to lepeat the factonng seveial times to obtain good estimates of 
the communalities and an acceptable factor matnx 

Uniqueness of communalities 

We have consideied the pioblem of aaceitaming whethei there are enough 
independent correlation coefficients in R to determine the factoi matnx for 
r factors This pioblem is solved by the relations of the inequality (10), 
which gives the maximum number of common factors that can be deter¬ 
mined fiom the mtercoi relations of n tests. In the theorem for the inequal¬ 
ity we stated that for any given number of common factoi s (r) to be deter¬ 
mined by a given number of tests ( n ) it is necessary that the inequality be 
satisfied The theorem does not state that the satisfying of the inequality 
(10) is sufficient to insuie that r factois can be deteimmed from the mtei- 
correlations of n tests In denvmg the inequality we merely counted the 
number of independent parameteis in the correlation matnx and the num- 
bei of independent parameteis in the desired factor matnx The relation 
between these two counts does not insuie that the diagonal values will be 
positive numbers between 0 and +1, and it does not take into consideration 
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the possibility of multiple solutions with real or imaginary diagonal entries 
m the correlation matrix 

If a correlation matrix is factored by the diagonal method with unit com- 
munalities (chapter 11 ), a factor matrix can be obtained which accounts for 
the intercorrelations exactly with not more than n factors If n factors are 
required to account for the mtercorrelations, then the last column of the 
factor matrix contains only one entry Hence it may be regarded as a spe¬ 
cific factor for the last test. We therefore have the theorem that there ex¬ 
ists at least one set of communalities between 0 and +1 for which the rank of 
the reduced correlation matrix is less than the number of tests. Every correlation 
matrix can be accounted for by not more than (n — 1) common factors 

In making an exact solution for the communalities, one would take a 
trial lank r and then expand minors of older (r+1) which should vanish 



Figure 1 

These mmois should be so selected that they contain a minimum number of 
diagonal cells In Figure 1 we have represented the correlation matrix and 
the puncipal diagonal Let ABCD be a minor which is so placed that it 
contains a minimum number of diagonal cells Let the number of col umn s 
which do not contain any pait of the minor be denoted b The number of 
columns b then is 

n — r — 1 , 

and hence the number of diagonal cells outside the indicated minor is 

2(n - r - 1) , 

so that the minimum number of communalities M that can be placed in the 
nunoi is 


(31) 


M = n — 2 (n — r — 1) 
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or 

(32) M - 2r — n + 2 

When M < 0, a number of minors of order (/'+1) can be taken which contain 
a particular communality and no otheis If M< 1, then a minor can be so 
placed that it will contain only one communality By expanding such a 
minor and setting it equal to zero, we have a linear equation with one un¬ 
known, namely, the desncd communality, which can then be numerically 
deteimined We then have 

(33) r < f 01 M < 1 

The numbei of factors is then less than half of the number of tests When this 
condition obtains, there is a single analytic solution fiom the selected 
minor If the communality so detei mined should exceed unity, then the 
rank of the correlation matrix must be assumed to be higher than r and a 
higher trial value for the i ank must be taken. When r is taken so that 
M< 0, the consistency of the several possible determinations of the eom- 
munalities must be examined 

Multiple solutions foi the communalities are of special interest, and they 
may arise when r is taken so large that the minor ABCD in Figure 1 con¬ 
tains two or moie communalities If we set M = 2, we have r = n/2, so that 
the minor ABCD of order (r+1) contains two communalities When this 
minor is expanded and equated to zero, we have a quadratic which may in 
some cncumstances give two real solutions for the diagonal value between 
0 and +1 

This problem has been investigated by E B Wilson and Jane Worcester* 
m several papeis They exhibit as an example a correlation matrix with 
n = 6 and r = 3 Their example is reproduced in Table 11, which shows the 
correlation matrix and two factorial resolutions, both for three factors and 
with diffeient sets of communalities In Table 12 we have another example 
from the same authois, which i educes to only one set of communalities A 
second solution gives diagonal values that do not he between 0 and +1 A 
thud example from the paper, “The Resolution of Six Tests into Three 
General Factors,” is shown m Table 13, for which the authors find no set of 
communalities between 0 and +1 that ieduce the lank to 3 A higher rank 
must then be assumed for this example It is pointed out by the authors 

* E B Wilson and Jane Worcester, “The Resolution of Six Tests into Three General 
Factors," Proceedings of the National Academy of Sciences, XXV, No 2 (Febiuary, 
1939), 73-77, and “Team Tests for Generals and Specifics,” ibid, , No 7, July, 1939, pp 
358-64; also, E B Wilson, “Review of Crossroads m the Mind of Man (by T L 
Ifelley]," Journal of Genet al Psychology , II (1929), 153-74, esp 156 



Table 11 


Correlation Matrix Factor Matrix 
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h* 
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48 

24 

64 

8 

0 

0 

64 

2 

56 
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66 

51 

86 

7 

6 
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3 

16 

20 
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18 
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23 
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1 
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30 

72 

.6 

4 
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56 
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64 

86 

23 

72 
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Alternative Factor Matrix 


I' 

II' 
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6524 

0 

0 
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4068 

0 
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- 0259 
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4774 
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386667 
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0441 

1835 
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Factor matrix unique 


Table 13 

Correlation Malnx 
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(“Team Tests for Generals and Specifics,” p 362) that, when the number of 
tests is much larger than the number of factors, then M is negative, and the 
question of multiple solutions does not arise 

It is of interest to note the number of independent conditions in the cor¬ 
relation coefficients for the resolution of n tests into r common factors This 
is the difference between the number of independent correlation coefficients 
and the number of independent parameters in the factor matux It can be 
obtained from the inequality (7) Denoting the numbei of independent 
conditions m the correlation coefficients T, we have 

, rv n(n—l) , r(r-l) 

(34) T = -nr + 2 , 

which has been written by Wilson and Worcester (in “The Resolution of 
Six Tests . . ” and “Team Tests for Generals and Specifics”) in the foim 

(35) T = \{n— r) 2 — h(n+r) 

When the number of independent conditions in the con elation coefficients 
is zero, the resolution is unique in the sense that no arbitrary paiametei is 
involved m the solution but not necessarily m the sense that theie is only 
one solution In the numerical examples of Tables 11, 12, and 13, T- 0, 
since n = 6 and r = 3; but M = 2, and hence multiple solutions may anse In 
one example the solution is unacceptable because of diagonal values that 
exceed unity 

These relations are of importance m dealing with factor pioblems in 
which the rank is as high as half of the number of tests If the test battery 
can be so constructed that the rank is not more than one-fourth or one- 
third of the number of tests, then the question of multiple solutions for the 
communalities does not arise. 

The uniqueness of communalities is discussed further m a later section 
of this chapter 

Adjustment of communalities for each factor 

In factoring a correlation matux, one may adjust the communalities 
after each factor if residuals are computed The simplest method of making 
this adjustment is to record m the diagonal entry the highest correlation oi 
residual of the column If the highest coefficient of the column is negative, 
one recoids this value in the diagonal with positive sign This piocedure is 
useful for large correlation matnces, and it can be used as a first approxima¬ 
tion for the factoring of a small table The procedure will be illustrated by a 
numerical example 

In Table 14 we have a small correlation matrix of order 5X5 and rank 2. 
Although method No 4 would be used only as a first approximation in fac- 
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toring a correlation matrix as small as this, it will be used here as a numerical 
example of the effects The true communahties are shown m the diagonal 
cells of li The first factor matrix was obtained with the group centroid 
method, using the known communahties The second-factor residuals were 
then identically zero, and the obtained communahties were the true ones, 
as shown in the table 

The second factor matrix was obtained by using the highest coefficient in 
each column as the estimated communality The discrepancies for a small 
correlation matrix are then often marked. In the fifth column, for example, 
the highest coefficient in the column is 72, whereas the true communality 


Table 14 

Correlation Matrix of Rank 2 
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( 64) 

56 

16 

24 

72 

2 

66 

( 65) 

38 

49 

67 

3 

.16 

38 
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48 

24 
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48 
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67 

24 

34 
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Factor Matrix Computed mth Factor Matrix Computed with 

Known Communahties Unknown Communahties 



I 

IX 

A* 


I 

II 

A* 

Discrepancy 

1 

680 

- 421 

640 

1 

703 

- 447 

694 

054 

2 

806 

- 029 

650 

2 

812 

- 029 

660 

010 

3 

486 

404 

399 

3 

510 

442 

455 

056 

4 

624 

437 

580 

4 

598 

400 

518 

- 062 

5 

817 

- 390 

820 

5 

788 

- 375 

762 

- 058 


is 82 Using these estimates, we obtained column I of the second factoi 
matrix by the group centroid method The correlational residuals were com¬ 
puted, and the diagonal entiies were adjusted by recording the highest resid¬ 
ual in each column The second factor column II was then obtained In 
the last column are sliown the resulting discrepancies between the obtained 
communahties and the true values. It will be seen that the discrepancies 
are smaller, the largest discrepancy being 06 For large correlation ma¬ 
trices the discrepancies between obtained and true communahties are still 
smaller 

In Figure 2 we have the true configuration of the five test vectors ob¬ 
tained from the first factor matrix of Table 14 . These test vectors are repre¬ 
sented by solid lines in the figure In the same figure we have superimposed 
the configuration obtained with the estimated communahties. These vec- 
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structure if it exists m the domain The same plan will also make it almost 
certain that the computing formulae, including the collinear graphic meth¬ 
od, will be applicable in the estimation of communalities at the start so that 
the investigator is free to use any factoring method, including those which 
avoid the computation of residuals The most troublesome correlation ma¬ 
trices are those which were not planned for factor analysis. In such cases the 
rank is often a considerable fraction of the order, and the computational 
problems then become more difficult 

Further study of linear dependence 

Since this chapter was written, the method of estimating communalities 
by linear dependence of lows and columns (method No 5) has been investi¬ 
gated further, and two papers have been written on this problem by A A 
Albert, of the Department of Mathematics at the University of Chicago * 
The method of estimation of communalities by linear dependence of rows 
of R will be described here with computing formulae and a numeiical ex¬ 
ample 
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Figure 3 


In Figure S we have represented the correlation matrix, partitioned as 
described earlier in this chapter The symmetric submatrices, R 22 , and 
R33, are here defined with zero diagonal elements The corresponding diag¬ 
onal elements are given in the diagonal matrices D h and I> 3 It will be 
assumed that the correlation matux has been inspected for clusters and that 
the number of distinguishable clusters represents a good estimate of the 
lank In assigning tests to the three groups into which the correlation ma¬ 
trix is sectioned, it will be assumed that one test from each cluster has been 
assigned to the first group and that one test from each cluster has been as- 

* "The Matrices of Factor Analysis,” Proceedings of the National Academy of Sciences, 
XXX, No 4 (April, 1944), 90-95, and “The Minimum Rank of a Correlation Matrix,” 
ibid., No. 6, June, 1944, pp 144-48 
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signed to the second group The result will be that the submatrices, fin, 
R«i, Ru, and R 22 , are square and of older r where r is the number of clusters 
The matrix Rn is square and of order (n—2r) The lemaming submatrices 
may or may not be square The underlying assumption is that if all the 
factors aie represented in each of the first two groups, then the correlations 
of the thud group can be accounted foi as linear combinations of lows of 
either the first or the second gioup This is the centi al idea of this method of 
estimating communahties If a doublet clustei is found consisting of two 
tests which show unusually large coi relation, then one test from the doublet 
is assigned to group 1 and the other test of the doublet is assigned to group 2 
A doublet is fundamentally indeterminate as to factorial analysis, and it ap¬ 
pears as an ambiguity m the rotational problem 

When the con elation matnx has been arranged in this manner and as 
shown in Figure 3, we can express the rows of Rn as linear combinations of 
the lows of Rn The transformation can be written 

(36) AR 2 i = R 3 i , 

fiom which we can determine the transformation matnx A, since all the 
elements of the two submatrices are known and contain no communahties 
We then have 

(37) A = Rai R- 2 i 1 

It will be found that R21 is the only inveise that needs to be computed If it 
is found that R 21 does not have an inverse, then either the assumed rank is 
too high, so that its determinant vanishes, or the clusters have been chosen 
inappropriately 

When the transformation matrix A has been determined, we express the 
rows of Rn as linear combinations of the rows of R 33 by the same transforma¬ 
tion, and we then have 

(38) ARn — Rzz Dz 

The side conelations of this matnx must agiee with those of the subma¬ 
trix R:u If this is not the case, then the rank has piobably been assumed 
too low If this agreement is obtained, then the diagonals are those of R s3 
The communahties of I2 33 are then known 

Applying the same transformation to Rn, we have 

(39) A(Rn + D») = R 32 , 
or 


(40) 


Raz — AR% 2 — AD' 
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We compute the left membei of this equation. The ratio of each column 
sum to the corresponding column sum of A is then the desired communality 
for the corresponding column of R 22 . * 

The same process is earned out for the remaining set of thiee submatrices 
The rows of Raa can be expressed as linear combinations of the rows of I( VI) 
and wo then have 

(41) BR 12 = Ra 2 , 
so that 

(42) B = Rzz Ri% • 

We can wnte this equation m terms of the mveise alieady computed, and 
we then have 

(43) B' = Ri i 1 R m , 
so that 

(44) B = (Rz i 1 RuY 
Applying this transformation to Rn, we have 

(45) B(Ru + Di) = R 31 
01 

(46) BRu + BDi = R 31 , 
so that 

(47) BD 1 = (Rn - BRu .) 

The ratio of the columns of the right member to the columns of B give the 
corresponding estimates of the eommunalities of Bn 

Table 15 shows the computations for this method of estimating com- 
munalities which is illustrated on a coi relation rnatnx of order 6 and rank 2 
The summarized computations follow the outline as shown 

A limitation of this method for experimental data is that the rank is 
never exactly r where r<n and that the determinations will therefore be ap- 

* In practice one can save computational woik by finding only the column sums be¬ 
cause these enter into the deteimmation of elements of D 2 
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proximate Even when the appioximate rank r is adequate, as far as corre¬ 
lational lesiduals are concerned, the present method may, nevertheless, 
show discrepancies in the estimated communalities for experimental data 
The method has not yet been tried extensively enough on experimental data 
to show whethei it is prefei able to the simpler methods of estimation The 
method is certainly amply justified foi correlation matrices in which the low 
r ank is exact or very closely approximated 

In the two papers on the communahty problem Albeit defines the ideal 
rank of B as the largest order of a non-vanishing minor of R which does 
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not contain diagonal elements He proves the interesting theorem that if 
the ideal rank r of R is also the rank oj the submalnces Rai and R 32) then there 
exist unique communahties such that the resulting correlation matrix R has 
rank r This is an encouraging finding because it has been assumed m the de¬ 
velopment of multiplc-factoi analysis that there exists a unique set of com¬ 
monalities which are consistent with the expenmcntally given off-diagonal 
coi relations The existence of such a set of communahties merely m amtams 
the lank of the off-diagonal entiles, which Albert has called the ideal rank 
of R It had previously been pointed out by E B Wilson that if the number 
of tests is much larger than the number of factors (n> 0, then the question 
of multiple solutions for the communahties does not arise 

Experimental comparison of communality estimates 

Since this chapter was set in type Mi Fiank Mcdland has completed a 
Master’s dissertation, entitled “An Empirical Comparison of Methods of 
Communality Estimation ” Nine methods of estimating communahties 
were tiied on a 63-vanable correlation matnx from Psychometric Mono¬ 
graph No 2 These communality estimates weie compared with com- 
munahties that were computed by the centroid method, with diagonal 
adjustment after each factoi Medland found that the centroid formula 
of equation (15) gave the most consistent communality estimates Prac¬ 
tically the same results were obtained by the giaphical method that is 
represented by equation (30) Since the centioid foimula of equation (15) 
is computationally the more convenient of these two methods, it may be 
consideied the preferred method 
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THE SIMPLE-STRUCTURE CONCEPT 


The score function 

Simple structure has been briefly mentioned in previous chapters, but it 
will be considered here m more detail as to its significance and implications 
The principles of simple structure are fundamental m making factor analy¬ 
sis a scientific method rather than merely a method of statistical condensa¬ 
tion, and it is therefore one of the central themes m factonal theoiy Some 
writers on factorial theoiy have not yet accepted this concept as a poweiful 
aid in exploiatory factorial investigation It is surely not the only way to 
resolve the pioblem of finding a significant reference frame, but it has so 
far yielded some very illuminating results 

Consider a set of measurements S„ of individuals i m traits j The meas¬ 
urements selected for factorial study are cori elated, and hence they can be 
assumed to have some parameters in common Let the measuiements be 
expressed in terms of these parameters by the equations 

Sn = %2 lr . , xpzj , X /,) , 

X2if t %pit t X,t) , 

Sji — 4>j(Xlt) a2,, , Xptj , Xn ) , 

Snt = *T2t, , Xpu t Xn) 

Here we have each measurement s,, for individual i expressed m terms of r 
paiameters x pt! which are descriptive of individual i The functions 0, are 
in general non-linear, and they are unknown They may be assumed to be 
unique for each measurement j If a measurement S, does not depend on a 
parameter X„, then that parameter is absent in the function 4>j Every pair 
of measures, j and fc, will be assumed to have one or more parameters m 
common if 0. If r,j,= 0, they may or may not have parameters m com¬ 
mon 

For any given list of n measurements S„ let there be q parameteis in¬ 
volved Some of these parameters will be unique in that they are not in¬ 
volved in two or more of the score functions S, We limit the analysis to the r 
parameters which are involved m two or moie of these scoie functions and 
which determine correlation between the scores. It is this set of r parameters 
that determines the side correlations and the intercom elations r,k, where 

319 
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) 7 *k The (q~r) unique parameters influence only the diagonal correlations 
r 3 „ which are eliminated from the analysis in order to make the problem 
soluble. If we knew all the parameters and all the functions tfo, we should 
probably be able to design a set of measurements for any set of paiameters 
so as to make a determinate solution, lnespective of whether the parameters 
were common or unique. Bub when the parameters become known, there 
will be no provocation for factor analysis It is our ignorance about the 
parameters and the score functions that prompts us to use the exploratory 
factorial methods In our present state of knowledge we are likely to add 
unique parameters with every new measurement, so that we are likely to 
have q> n, a situation that involves more parameters than there are meas¬ 
urements for each individual 

Any particular list of r paiameters defines a domain, which includes all 
possible measurements that are determined in whole or in part by these param¬ 
eters In practice, a domain is defined by a field of related measurements, 
since the underlying parameters are unknown A domain is described in 
practical work in terms of the content and nature of the measurements, such 
as auditory perception or visual illusions or speed peiformances The com¬ 
plexity of a measurement S, is the number of parameters X v that it involves 
and which it shares with one or more other measurements in the battery 
Hence the complexity of a test or measurement is dependent on the test 
battery in which it is being analyzed The complexity of a test j is, in pait, 
dependent on the test battery When we speak of the complexity of a single 
test apart from a test battery, we refer to the total number of parameters 
in the score function S, 

Consider a table of order n X r, in which each cell represents a parametei 
for a particular test. Each low represents a function and each column 
represents a parametei The complexity of a battery of n tests is r, and it 
should be the aim of the investigator to choose a set of n tests or measure¬ 
ments for the domain which he is studying so that n>r Ilis scientific in¬ 
tuitions and hypotheses are his chief guides here If the score functions were 
known, we could mark an x on each cell (jp) to show that a parameter X„ is 
involved m the score function 4> n and we could leave blank each cell (jp) in 
which the parameter X p is absent in the score function <t> 3 We call such a 
table a factor pattern (see chapter ix for different types of factor patterns) 

If an analysis is to be made by the principles of simple stiucture, then the 
investigator gambles that the complexity of each test or measurement is 
less than the complexity r of the battery as a whole He hopes that the test 
complexities are considerably less than that of the batteiy If the test com¬ 
plexities are less than r, then there will be one or more blank cells m each 
row of the factor pattern Such a pattern will be considered as evidence of 
an underlying simple order in which each parametei or process m the domain 
is not involved in every test measurement If the parameters are uncorre- 
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lated m the experimental population, then the pattern will be indicative of 
an orthogonal simple order for that population If the parameters are coire¬ 
lated m the experimental population, then the pattern indicates an oblique 
simple order The question of whether the parameters aie correlated 01 un- 
correlated among the subjects who happen to be used for studying the do¬ 
main is, in. general, a secondary matter when compared with the scientific 
problem of discovering an underlying older m the domain 

The score functions S, may be represented by a surface, as in Figure 1, 
In this figure we have considered only two of the parameters, X i and Z 2 , as 
well as the score S, If the score S, is independent of X 2 , then the surface 
S, will be a cylinder parallel to X 2 (A cylinder is, of couise, not necessarily 
circular m section I is a surface parallel to an axis.) An individual with 
parameters xu and x 2l will have a score s,, as shown, and this score will be 



independent of the parametei X 2 This reasoning can be extended to n di¬ 
mensions If the surfaces could bo constructed or otherwise represented, 
the cylindrical forms of the surfaces could be taken as clues for locating the 
reference frame, especially if a number of such suifaces were found which 
defined the same axes m this manner If a reference frame could be de¬ 
termined by the cylindrical surfaces, it could be expected that the axes, so 
determined, would represent parameters for which some physical or opera¬ 
tional interpretation could be made It would not necessarily follow that 
such a reference fiame would be the only feasible one, but the simple order 
indicated by these axes should be illuminating in the exploratory study of a 
domain for which no hypotheses are available Each axis should represent 
some underlying process, pai ameter, or category which may ot may not have 
been suspected in setting up the test batteiy Unfoitunately, these surfaces 
are not immediately available in the measurements and their relations, so 
that approximations must be used. 
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There are certain characteristics of the surfaces Sj which may be sur¬ 
mised by psychological considerations. Most of the factorial studies have 
been made with those mental traits which can be called “abilities ” An 
ability may be considered to be any trait which describes how well a person 
can do something It has been observed that all such traits have correla¬ 
tions that are positive or zero, and this seems to be the rule for the most 
diverse abilities The parameters that underlie individual differences in the 
mental abilities may be tentatively assumed to be of such a nature that they 
are covariant with the overtly manifested abilities If that should be found 
to be acceptable, then we should be able to infer that the surface S, rises 
with an increment in any of the underlying parameters X p By these psy¬ 
chological considerations we should infer that 


( 2 ) 


aSj_ > 

ax t - 
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for those traits that are classified as abilities In other woids, the surfaces 
S, for mental abilities may be assumed to be monotonic increasing functions 
of the parameters X„* If this inference is acceptable, then it should be pos¬ 
sible to represent all the surfaces S, reasonably well with second-degree 
equations for the mental abilities, barring what might seem now to be 
strange irregularities or discontinuities in the second derivatives 

While these characteristics of the surfaces S, may seem plausible for the 
mental abilities, there do not seem to be any similar obvious restrictions for 
the surfaces that represent temperamental traits If some of these surfaces 
are characterized by conspicuous maxima and minima, then the observa¬ 
tion equations for them might have to be of the third or higher degree 
Gross discrepancies and large residuals may then be found in the ciude 
linear assumptions on which piesent-day factorial methods are based Most 
of the statistical methods in current use involve the same assumption. 

The various rating schedules for personality traits are frequently ar¬ 
ranged with special reference to social acceptability and success rather than 
with reference to the degree of some underlying trait Such ratings may 
give trouble m factorial studies if the relation between the amount of some 
underlying trait and the rating on some kind of social acceptability has 
a maximum for an optimal amount of the underlying trait The assumption 
of linearity between the underlying parameters and the overtly observed 
social acceptability is then violated more seriously than if we were dealing 
with personality descriptions that are mo no tonic inci easing oi deci easing 

* If time per unit task is used as a score, then the score function is decreasing 
monotonia, m which case it can be reversed so that all correlations become positive This 
applies to abilities and not to all psychological measurements 
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functions of the undei lying parameters An example may clarify these re¬ 
strictions Let us suppose that one of the impoitant underlying parameters 
in describing a man is the rate at which he moves generally A person with 
a very low value of this parameter would be described as slow, inert, and un¬ 
responsive The upper extreme of this paiameter would be described as a 
personality that is jerky, scatteibramed, nervous, irritable, and perhaps 
unstable An optimum value between the extremes is likely to be associated 
with more favorable descriptive categories as regards social acceptability 
and success The sketches of Figure 8 are drawn to show that a monotonic 
increasing or deci easing surface can usually be fitted better by a linear ap¬ 
proximation than can a suiface with conspicuous maxima or minima 



In Figure 3 we have represented two plane surfaces which may be thought 
of as best-fitting planes for the scoie surfaces S, andS fc The score surface S, 
of Figure 1 represents law scores Such a surface also could represent stand¬ 
ard scoies by merely moving the origin to the level of the mean and adopt¬ 
ing a vertical scale with the standard deviation as a unit of measurement 
With this translation of the origin and with the stretching factor, we have 
the planes passing through the origin as shown m Figure 3 It is of some 
interest to note, m passing, that the rank of a matrix of raw scores is reduced 
by 1 when the scores are modified to deviation scores about the mean. 

The two vectors J and K in Figure S are unit normals to the planes S, and 
St, respectively The direction cosines of each normal are also the coeffi¬ 
cients of the equation of the corresponding test score plane If a surface S, is 
cylindrical and paiallel to some axis X p , then the best-fitting plane for that 
surface is also parallel to that axis except for the accidents of sampling. 
The direction cosines of the normal to the plane will have a zero-direction 
cosine for the axis X p The vectois J and K are unit test vectors in the total 
test space If Xi and Xi are correlated and if a number of test vectors are 
orthogonal to them so that these axes are chosen for the oblique factorial 
matrix V, p> then the test vectors will have zero projections on these axes 
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and the corresponding entries m the factor matrix will be zeros In this 
manner we see that if a numbei of test scoi e surfaces S } are cylindrical and 
parallel to each other, then the corresponding axis X P is chosen as a refer¬ 
ence vector and is represented by a large number of zeio entries in a column 
of the oblique factor matux 

The relations of Figure 1 may be used to illustrate another fundamental 
principle in factor analysis as regards the scientific interpretation of a simple 
structure when it is found Suppose that there are individual differences in 
the ability to visualize space A test battery could be assembled, including 
visual tests that do not involve this factor and other visual tests that do in¬ 
volve it The factorial separation would sustain the hypothesis that such a 



factor is distinguishable among visual functions If a parameter or factor or 
function exists which becomes opeiative in some way m the visualizing of 
space and not m other visual tasks, then we should expect those scoie sur¬ 
faces to be parallel to an axis which does not involve the particular paiam- 
eter If the separation of the score suifaces actually agrees with the classi¬ 
fication of the tests as including or excluding the particular function, then 
the factorial results sustain the hypothesis 

The pnnciple of interest here is that the identification of a factor 01 
parameter does not presuppose the experimental population to be m any 
sense lepresentative of any sort of hypothetical general population In fac- 
tonal studies that are made for the discoveiy of an undeilymg older m a 
domain, the best proceduie is not to select a random group of subjects, 
but rather to select the subjects so that their attributes are as diverse 
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as possible m the domain to be studied Two or three freaks in the charac¬ 
teristics of interest are worth more than fifty average subjects This prin¬ 
ciple is not new If you wanted to inquire mfoimally about the existence of 
the ability to visualize space, you would gam most by selecting some drafts¬ 
men or engineers and some people who evidently avoid visualizing tasks and 
methods of thinking Later, when the factor had been identified clearly on 
exceptionally gifted or deficient individuals, you could proceed to the task 
of establishing norms to determine the actual range of ability in random 
samples of the general population Here, again, we see how the puipose of 
a factorial study determines the kind of subjects to be selected for the ex¬ 
perimental group If we have no hypotheses about the kind of people who 
differ fundamentally m the domain to be studied, then we proceed by in¬ 
viting any conveniently available gioup of subjects in the hope that they 
will differ markedly among themselves m the several parameters or factors 
of the domain to be investigated If a few subjects can be found who are 
known to differ widely in the domain, they should be included No assump¬ 
tion of normality of the distribution is involved in factorial analysis If we 
could find bimodal distributions for some tests and factors, they would be 
especially interesting, provided, of course, that the bimodalities were not 
artifacts The question of whether the conelations between the tests m an 
expenmental population are in any sense lepresentative of a general popu¬ 
lation is lirelevant Nor does it mattei whether the correlations for one ex¬ 
perimental group differed markedly from the correlations of the same tests in 
another experimental group If the factorial analyses aie made independ¬ 
ently, the same factois should be identified, but the numerical values would, 
of course, be different if the groups differ widely The correlations for each 
group are determined by the factors involved m the tests and by the corre¬ 
lations of the factors in each expenmental gioup 

We have defined the domain as the field of all possible measurements 
that are determined, at least m part, by a set of parameters In exploratory 
studies the domain is defined tentatively m terms of the content 01 nature 
of the tests of a battery The principal purpose is to discover the paiam- 
ctcis or factois and something about the nature of the individual differences 
that they produce The individual subjects aie examined, not foi the pur¬ 
pose of learning something about them individually, but ratbei for the pur¬ 
pose of discovering the underlying factors Ultimately, we want to be 
able to appraise each individual as to each of the factors, but this problem 
raises certain other questions about the domain 

In setting up the fundamental linear observation equation foi factoi 
analysis, the individual score s,, is expressed m terms of the test coefficients 
c im and the individual scores m a set of arbitrary orthogonal factors x mt By 
rotation of axes the scoies can be expressed as linear combinations of indi- 
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vidual scores x pl in the primary factors p instead of the arbitrary factors m 
Some writers have attributed to factor analysis the assumption that all the 
subjects in an experimental group use the same factors m doing a test, but 
such an assumption is not made in factor analysis Suppose that a task can 
be done in one of two ways that involve either of two factors, 1 and 2 Sup¬ 
pose, further, that some subjects use factor 1 and that other subjects prefer 
factor 2 in doing this task If this test is included in a battery for factor 
analysis and if the battery contains other tests by which factors 1 and 2 
can be clearly differentiated, then the test in question will have a complexity 
of at least 2 We should then infer that the test has these two factors, m 
that v 3 i and v,i are non-vanishing m the factor matrix V; but it would not 
follow that each individual subject used both of these factors m doing the 
test To decide whether this is true, one turns, first, to introspection m 
doing the task to ascertain whether it seems psychologically necessary that 
both factors should be simultaneously used, or whether it is a question for 
each subject to decide which method to adopt in doing the test Here one 
must be careful not to be biased by one’s own preferred ways of doing 
things In case of doubt, we ask a number of subjects how they did the task, 
including the instiuetions and fore-exercise, and to describe any useful 
tricks or dodges that they may have discoveied in doing the test. These in¬ 
formal reports are sometimes very illuminating A little expenenco of this 
kind soon emphasizes the importance of introspection as compaied with 
casual judgments about the factorial composition of a test merely by a su¬ 
perficial inspection of its content For example, a syllogism test about the 
relative statures of three men was found to have a significant saturation on 
the space factor, although the test was entirely verbal in appearance. It 
was found that some subjects, but not all, used simple diagrammatic repre¬ 
sentation of the premises such as “S—J” to represent the premise that 
Smith is taller than Jones, so that the inference was also spatially repre¬ 
sented In order to make the analysis more complete, one might separate 
the subjects into two groups according to preferred methods of doing a test 
and reanalyze the results In such a situation we should expect to find a 
different factorial composition of a test for the two groups of subjects 
Another situation in which the factorial composition of a test can be 
altered is that in which the subjects are given instruction in doing a task 
The factorial composition can be markedly changed, since the test is itself 
modified by the instruction, even though the test form remains identical 
after instruction After all, the psychological nature of a test is determined by 
what goes on in the mind of the subject and not only by what is printed on the 
test papei In the same manner the factorial composition of a test is changed 
with the age of the subjects Sometimes we have arranged the older of the 
tests in a battery with refeience to the suggestive effects of one test on a 
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following test, especially if one test lequires a mental set similar, or oppo¬ 
site, to that of a previous test. Fortunately, these ovei lapping effects are 
not often serious 

When it comes to the pioblem of using the tests, not foi the purpose of 
identifying mental factors, but for the purpose of appraising the factors in 
each individual, we resort to the same methods that are in current use with 
psychological tests. The best that we can do is to determine the correlation 
of each test with each primary factor, r ]p . These are the validity coefficients 
of the tests with respect to the pnmaiy factors We select several tests 
whose validities are the highest available foi a particular primary factor 
p These several tests are pooled in a composite The composite score is 
then used as the best available estimate of the individual’s score x pi m the 
piimaiy factor p If each test has saturations in several factors, such as 
(1, 2), (1, 3, 5), (1, 4, 7), then we try to distribute the distracting saturations 
foi other factors so as to emphasize the validity for one of the factors The 
same result can be obtained by multiple conelation analysis in the usual 
ways where r„ aie validity coefficients which should be studied with the 
intei correlations of the separate tests 

Simple structure in r dimensions 

The simple-sti ucture piinciple will be lestated here for r dimensions 
Some of the previously given definitions will be repeated in this context * 

The r orthogonal refeience vectors that are implied in the orthogonal fac¬ 
tor matnx F aie called orthogonal reference vectors These vectors are re¬ 
garded as a fixed orthogonal frame in terms of which the vanous factonal 
lelations are expressed It is this fixed fiame in terms of which we describe 
the test vectors ] in the rows of F, the r hyperplanes of an orthogonal or 
oblique simple stiuctuie, and the pumary trait vectors These fixed orthog¬ 
onal axes are refei red to by the subscript m, or M and m if two subscripts 
aie desned Thus the test vector j has the projections a jm on these orthog¬ 
onal axes, as shown m the rows of F When this reference frame is deter¬ 
mined by some form of the centroid method, the frame will be called centroid 
reference vectors 

The unique configuiation of trait vectors defined by the conelation ma- 
tnx is called the correlational configuration or the trait configuration This 
is the configuiation descubed in teims of the fixed oithogonal fiame by the 
factoi matrix F The combination of the correlational configuration and 
any set of leference vectois is called a structure , and it can be seen that a 
structure is itself a configuiation of (n+r) vectors The problem is to sub¬ 
stitute foi the arbitiary leference frame another frame, orthogonal or 
oblique, which will have scientific intei pretation Each reference vector of 
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the new frame should represent a scientific category, a parameter, or a 
function m terms of which the trait eonfiguiation can be interpreted 

The r lineally independent reference vectors define the same number of 
co-oi dmate hyperplanes Each hyperplane is a subspace of dimensionality 
(r — 1), and it is defined by a reference vector. The subspace of (r— 1) di¬ 
mensions which is orthogonal to the reference vector A p will be called the 
co-ordinate hyperplane L p If a reference frame can be found such that each 
test vector is contained in one or more of the r co-ordinate hyperplanes, then 
the combined frame and configuration is called a simple structure. If the 
new reference vectors are orthogonal, the total configuration is called an 
orthogonal simple structure If a set of r hyperplanes of dimensionality (r — 1) 
exists such that each test vector is in one or more of the hyperplanes and if 
the normals A,, are oblique, then the total configuration of test vectors and 
reference vectors is called an oblique simple structure. It follows from these 
definitions that each test vector that contributes to the identification of a 
simple structure must be of complexity less than r. A configuration of test 
vectors which defines a simple structure will be called a simple configuration 

If an nXr factorial matrix is set up with arbitrary entries m all cells, 
there is, in general, no transformation by which each of the n variables can 
be described in terms of fewer than ? factors It is assumed that n is large 
in comparison with r Therefore, the appearance of simple structure in a 
factorial matrix derived from observation commands attention It is not a 
chance matter When found in experimental data, it reveals order within 
the « variables, in that r<n categories are required for describing them col¬ 
lectively and fewer than r categories are required for each one of them 
separately 

If an underlying physical order of the n tiaits is such that each of the 
traits can be described in terms of a smaller number of factors than are re¬ 
quired for describing the traits collectively, then the underlying physical 
order will be called a simple order 

If a simple order exists for a set of n traits and if the r factors are statis¬ 
tically independent m the experimental population, then the corresponding 
physical order will be called an orthogonal simple order Hence the configu- 
lation which represents an oithogonal simple order among the n tiaits is an 
orthogonal simple stiucture An order among the traits involves, of course, 
not only the traits themselves but also the categories in terms of which they 
are described These categories are themselves traits which may or may not 
be experimentally isolable 

It is useful to summarize the several fundamental concepts m this anal¬ 
ysis The concept order refers to the relation between the traits and the 
categones in terms of which the trails are to be described and compre¬ 
hended The correlational matrix describes merely the relations among the 
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traits, independent of the descriptive categories The factorial matrix de¬ 
scribes the traits or variables m terms of a set of descriptive categories The 
trait configuration is a geometrical representation of the correlational ma¬ 
trix, and hence it is also independent of the descriptive categories The 
structure is a configuration which represents not only the traits but also the 
descriptive categones The scientific problem is essentially a seaich for a 
set of descriptive categories in terms of which oui conception of the traits 
or variables is the simplest possible If an overdetermined simplicity in our 
conception can be achieved, then the traits or variables will be said to re¬ 
veal a simple order The search for these categories has its direct analytical 
counterpart m the search for a set of reference vectors which will reveal a 
simple structure A simple structure is a configurational representation of a 
simple order If the simplifying descriptive categories happen to be statis¬ 
tically independent m the experimental population, then the trait configuia- 
tion can be so rotated in its arbitrary orthogonal fiame that each trait vec¬ 
tor is contained m one or more of the r orthogonal co-ordinate hyperplanes. 
The result is an orthogonal simple structure, and the reference vectors repi e- 
sent a set of statistically independent traits that serve the simplest possible 
comprehension of the given traits or variables. 

Alternative simple structures for the same domain 

We should distinguish between uniqueness of the simple structure for a 
given correlation matrix and uniqueness of the descriptive parameters or 
concepts which the pumary axes represent In eai her chapters we have con¬ 
sidered examples, such as the cylinder problem and the box problem, m 
which the simple structure does not give the only meaningful set of param¬ 
eters or descriptive concepts for a configuration It is unlikely that unique¬ 
ness m that sense could be found in any domain Furthermore, it is con¬ 
ceivable that a domain could be represented by two sets of measures which 
would show two different simple structures, if the two sets of measures were 
taken m quite different ways Both sets of primary axes would then repre¬ 
sent underlying parameters for which meaningful interpretation could be 
expected The two sets of parameters could be regarded as alternative sets, 
and they should be revealing as to the nature of the underlying order 

The box problem, which can be easily visualized, has served to illustrate 
these principles Consider one set of measurements of the boxes which are 
simple functions of the three independent edges, as in the previous examples 
Consider also another set of conceivable measures of the same population of 
boxes in which each measure is some simple function of the basic parameters 
for volume, v, and the ratios f = s/z and u — x/y The thiee measures v, t, 
and u are superimposed on the simple structure x, y, and z in Figure 4- 
The basic measures x, y, and z might represent width, height, and depth, 
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respectively It is instructive to see what would happen with different sets 
of measurements of the boxes. First, consider the case in which we meiely 
add the three measures v, t, and u to the measures previously listed and 
which contribute to the simple structure of the box problem. The result 
would be merely the same triangular arrangement of the measures forming 
the triangle x, y, and z, and there would be the thiee additional measuies 
v, u, and f, which would not conti lbute to the identification of the simple 
structuie In the fiist example of the box pioblem wc had two such meas¬ 
ures, namely, 16 and 17, which represented the volume and the principal 
diagonal of each box. These did not participate m the simple structure 



If we should construct an entirely new set of measurements which were 
so composed that they were linear combinations of the three measuies 
v, t, and w, then they would define a new simple structure with piimaiy vec¬ 
tors v, t, and u The previous configuration x, y, and z would not be seen. 
If we proceeded to construct this new simple structure, we should have 
measures such as “one-half of the volume in cubic inches plus three-fourths 
of the ratio x/z” and perhaps twenty others similarly constructed We 
should then be able to show that the previous structure x , y, z was not 
unique for the domain The response of a scientist to such a finding would 
depend entirely on the manner in which the new structure v, t, u was ob¬ 
tained If the new structure had been obtained merely by picking three 
linearly independent measures in the first configuration and then arbi¬ 
trarily assembling linear combinations of them for a new battery, then, of 
course, the new structure would be regarded as an artifact, and it would 
demonstrate nothing regarding the underlying order for the domain If an 
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author were caught doing such a thing, his scientific honesty would be 
questioned unless he presented it merely as some sort of stunt The arbi¬ 
trary character of his battery of measurements would be questioned 
But there could be situations in which the new structure v, t, u would 
be valid and significant Suppose that these new parameters were signifi¬ 
cantly related to the manner in which the box shapes were generated or in 
which they weie used or interpreted. Then an mvestigatoi who assembled 
a battery of measurements to repiesent the domain might find the structuie 
v t, u, and it would serve him in studying the underlying order The as¬ 
sembled battery would reveal its organization in the new structure 

Let us suppose that two investigators found these two simple structures, 
using two different batteries of box measurements. Each of them might find 
a set of primary factors, and each might succeed in discovering some 
appropriate interpretation for his own set of three basic parameters 
Such a finding would not necessarily show that one of them was wrong 
They might have found two sets of alternative parameters for describ¬ 
ing different aspects of the domain in question The challenge is then 
to study closely the two sets of measures to see what different aspects 
of the domain they cover A new factorial experiment could be set up 
with both sets of measures in the same battery, and the configuration 
would then be like Figure 4 Additional factors might appear because 
the previous specifics might appear as common factors in the new analy¬ 
sis, but our present concern is about the principle involved in the possi¬ 
bility of alternative structures This would be a case in which there would 
be more hypcrplancs defined by the test vectors than there are dimen¬ 
sions in the common-factor space Such a situation should not be regarded 
as a failure in factorial analysis. It might reveal alternative sets of paiam- 
eters in the domain, which should all eventually be interpreted So far we 
have found only an occasional suggestion of secondaiy hyperplanes, but 
none of them have seemed to us to be sufficiently overdeteimmed to justify 
special investigation. 

In judging the difference between alternative structuies that should be 
sepaiately interpreted and the artifact which could be aibitrarily con¬ 
structed from any test batteiy, it is well to look at the nature of the meas¬ 
urements involved Both sets of measurements should be natural and ap¬ 
parently relevant measures descriptive of the domain in question If one 
set of measurements consists merely of a set of linear combinations of a 
few previous measures, 01 the equivalent as to arbitrary character, then its 
relevance to the problem of analyzing the domain may be questioned In 
any event, subsequent factorial study of the same domain should reveal 
whether a previously found structure is to be sustained 
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Uniqueness of primary factors 

One of the most fundamental problems m factor analysis concerns the 
uniqueness of the primary factors and their interpretation When a factor 
analysis has been completed with a clear simple structure, the first question 
is naturally “What are the parameteis or factors?” It seems to be in the 
nature of science that such a question has no unique answei. In scientific 
work a parameter is one of the measurable attributes of an object m terms of 
which it is described In general, one tne3 to choose a small number of 
parameters which define all the relevant attributes of the object The au¬ 
thor is successful if he finds a small number of easily comprehended param¬ 
eters that cover his problem Parameteis in scientific work and the con¬ 
cepts which they represent become the conventionalized chaptei headings 
in textbooks Even notation becomes a matter of convention A circle on 
a graph is defined by the two co-oidmates (x, y) of its center and by the 
radius r. These parameters aie easily understood and easily used in most 
problems, but, of course, they aie not unique For some pioblems they 
would be awkward, and another set of parameters would be chosen The 
same is true m scientific work The familianty of mass, momentum, and 
acceleration should not lead us to suppose that they are necessary absolutes 
for the description of objects m motion If these reservations about the 
uniqueness of parameters apply to physical science, it would seem wise to 
acknowledge them also in the seaich for descriptive parameters or function¬ 
al unities of mind. 

The choice of a set of fundamental concepts m terms of which any do¬ 
main of nature is to be comprehended is probably meaningless to nature 
The choice is for us to make in tenns that will attain intellectual control and 
consistency, as empirically determined, and by which this control can be 
attained, as far as possible, with ideas that are already familiar to us Fun¬ 
damental scientific concepts that are successful m this sense aie as social as 
other inventions To hunt for a unique solution in the comprehension of a 
set of related phenomena is an illusory hunt for absolutes It is probably 
safe to say that an apparently unique set of concepts in any domain is mere¬ 
ly the symbol of our immaturity in the exploration of that domain The 
recognition of the lack of uniqueness in scientific concepts does not imply 
that some sets are not more useful and fruitful than others Those param¬ 
eters are pi eferred which reveal the phenomena as of a simple underlying 
older 

Recognition of the fact that descriptive parameters are not unique has 
led some wnteis on factor analysis to give up the problem in defeat They 
remain content with a set of statistically determined parameters or factors 
which are wholly determined by the circumstances of the experiment at the 
moment The simple-structure concept was invented, if anything so simple 
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can be called an “invention,” as a compiomise with this pioblem It was 
the simple idea of finding the smallest number of 'parameters for describing 
each test. In numerical terms this is a demand for the smallest number of 
non-vanishing entiles in each low of the oblique factor matux It seems 
strange indeed, and it was entirely unexpected, that so simple and plausible 
an idea should meet with a stoim of protest from the statisticians This 
simple idea has turned out to be a much more poweiful analytical device 
than was at fast anticipated 

The pnmary factors that are identified by the simple-stiucture principle 
seem to be unique m the sense that they reappear in successive studies with 
different test batteries in the same domain. It is to be expected, however, 
that a primary factor may be found to be itself a complex when a part of a 
domain is investigated with large test batteries It will then give place to 
several new pnmary factors, which should retain their identity in successive 
test batteries that are adequate to isolate them As regards psychological 
mteipretation, it may be expected that this will also be modified or extended 
with repeated studies The number factor, for example, has been named so 
far only in terms of the numerical content of the tests, but this type of in¬ 
terpretation cannot be regarded as satisfactory The psychological inter¬ 
pretation should be made eventually m terms of psychological pi ocesses 
that will generally transcend test content except in those cases m which the 
test content and the psychological processes are descnbed by the same 
name In order for the interpretation of primary factors to be satisfactory, 
their psychological interpretation should be sufficiently clear so that the 
factorial composition of new tests can be predicted with confidence Prog¬ 
ress has been made in that direction, but this goal can be attained only after 
many factorial studies of the same domain. 

Analysis of a large number of factorial studies seems to indicate that those 
tests which are of low complexity and which, consequently, are the best 
indices of the nature of a primary factor are generally quite simple as judged 
by usual psychological considerations However, a test which seems simple 
without a knowledge of the primary factors may seem to be rather complex 
aftei the primary factors are understood Tests of high complexity, such as 
a mental age score, cannot be expected to contribute to the identification of 
primary factors. Job analysis m the future will be done in terms of known 
pnmary factors, so that counseling and employment procedures can be de¬ 
signed on rational grounds rather than, as at present, by a hit-or-miss hunt 
for acceptable validities for each particular task 

In the work of finding a set of fundamental psychological categories for 
the description of mental traits it should be possible to supplement the 
simple-structure principle by other principles In psychological work the 
choice of a set of basic parameters for describing mental traits could be fa- 
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eilitated by examining their genetic basis and by noting the 01 dei and the 
rate of development of the different functions Thus, foi example, the 
ability to do the Street Gestalt Completion test seems to be a function 
that matures very early, peiliaps as eaily as six or seven yeais, while the 
verbal factor seems to show continued growth thiough the high-school 
yeais The giowth of the veibal factor may be explained by the impel fec- 
tion of present tests, which aie markedly influenced by schooling and ex¬ 
perience 01 by slow maturation of the ability itself At any 1 ate, the pi unary 
factois which aie identified by the study of individual chffeiences at point 
age should be related to other findings on the late of giowth of the mental 
functions The primary factors should preferably be a set of paiameters 
which can be identified at each age by appiopnate tests The acceptance of 
primary factors in psychological investigation will depend on the extent 
to which they aie fruitful m non-factonal psychological reseaieh If they 
repiesent truly significant parameters for the description of mental traits, 
then they should also be revealed as distinguishable in the study of mental 
inheritance It is to be hoped that at least some of the primary factors that 
are now being identified in the study of individual differences at point age 
will also be found to be useful and simplifying pai ameters in the study of 
mental inheiitauce and in the study of mental giowth 

Uniqueness of simple structure in a given correlation matrix 

When reference axes have been found which produce a simple stiucture, 
it is of considerable scientific inteiest to know whether the simple stiucture 
is unique for the given coi relation matrix The necessary and sufficient 
conditions for uniqueness of a simple stiuctuie need to be investigated In 
the absence of a complete solution to this pioblem, five criteria will here be 
listed which probably constitute sufficient conditions for the uniqueness of a 
simple stiucture The scientific interpretation of the cell entries in the 
oblique factor matux V should not be attempted except after reasonable 
assurance that the simple stiucture of V is unique for the given configuia- 
tion If this assurance is not available, then the interpretation should be 
tentative 

When the factoi matrix has been rotated to an oblique simple stiucture, 
one can eliminate from consideration, as legards the uniqueness, those tests 
which have complexity equal to r Those tests are in the middle of the con¬ 
figuration, and they do not contubute to the location of the leference frame 
At the stait of the factorial analysis those tests are eliminated which have 
zero eommunahty, as indicated by the fact that then coirelations aie all 
vanishingly small Similarly, those tests can be eliminated fiom considera¬ 
tion of simple structuie whose communalities are low if they have not been 
eliminated at the start because of their low conelations In practice, one 
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need only eliminate fiorn consideiation those tests m the oblique factor 
matrix which have complexity i , because the othei eliminations are made 
loutmely m factoring the conelation matrix to save the unnecessary laboi 
of canymg some tests thiough the factoring when they cannot contribute 
to the analysis 

If we eliminate the tests of complexity r, the remaining tests should con- 
tiibute to the location of the leference frame We shall describe five useful 
catena by which the r reference vcctois can be determined These are as 
follows 

1) Each row of the oblique factor matrix V should have at least one zeio 

2) For each column p of the factoi matrix V there should be a distinct 
set of r linearly independent tests whose factor loadings v 3P are zero 

3) For every pan of columns of V theie should be several tests whose 
entiles v 1P vanish in one column but not in the other 

4) For every pair of columns of V, a large proportion of the tests should 
have zeio entiles in both columns. This applies to factor problems with 
four oi five or more common factors 

5) Foi eveiy pan of columns theie should preferably be only a small 
number of tests with non-vanishing entiles in both columns 

When these conditions aie satisfied, the plot of each pair of columns 
shows (1) a laigc concentiation of points in two radial streaks, (2) a large 
number of points at or near the origin, and (3) only a small number of 
points off the two radial streaks For a configuration of r dimensions there 
are \r(i —1) diagrams When all of them satisfy the thiee characteristics, 
we say that the structure is “compelling,” and we have good assurance 
that the simple structure is unique In the last analysis it is the appearance 
of the diagrams that deteimines, moie than any other criterion, which of 
the hypeiplanes of the simple structuie aie convincing and whether the 
whole configuration is to be accepted as stable and ready for interpreta¬ 
tion * 

The fiist criterion demands that each trait should be descnbable in terms 
of fewer categones than are required by the whole set of n traits It is con¬ 
ceivable that, m some cxpci imental work, one or more of the traits will be 
so complex as to require description m terms of all the factors that enter into 
the traits collectively For the purpose of isolating the fundamental cate- 
gones, these tiaits are not useful, and they should therefore be ignored 
The cntenon demands that the list of traits be long enough so that, after 
elimination of several traits of complexity r, enough traits of complexity 
less than r remain to determine uniquely both the trait configuration and 
the simple sti ucture This principle may be illustrated with psychological 

* Ever since I found the simple-stiucture solution for the factor problem, I have never 
attempted inteipretation of a factorial result without first inspecting the diagrams 
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tests If one of the abilities to be isolated should be number sense, then this 
pnmaiy ability should not be lequired m all the tests of a batteiy The 
same lestriction applies to each of the abilities that is to be isolated 

The second criterion seems to be essential for the following reason Each 
column p of V is determined by a hyperplane L v , A hyperplane through 
the origin is determined by (r — 1) trait vectors These trait vectors are 
contained in L p , and therefore they have vanishing entnes v, P m column p 
Theiefoie, there must be at least (? — 1) traits with vanishing entries m 
each column of V m order that the hypeiplanes may be deteimined Since 
the hyperplanes should be overdetermmed by the data, it follows that the 
number of vanishing entries m each column of V should equal or exceed r 
The third criterion is suggested by the fact that the r hyperplanes must 
be distinct If two columns of V contain the same vanishing entries and if 
these exceed (r — 2) m number, then, assuming the test vectors to be linearly 
independent, the two corresponding hyperplanes are identical The third 
criterion was wutten so as to insuie both overdetermmation and distinct¬ 
ness of the hyperplanes that define the columns of V 

When the fourth criterion is not satisfied, i e , when thei e arc no points 
at or near the origin for a problem of more than foui or five dimensions, we 
can be reasonably sure that the hyperplanes have not been coi rectly located 
This has been our experience with large and small factor problems The 
fourth criterion is of importance in factor problems of four or moie dimen¬ 
sions In a three-dimensional pioblem a point at the origin on the diagram 
for factors 1 and 2 would have to be a measure of only the third common fac¬ 
tor, so that its complexity would be 1 In two dimensions there can be no 
points at the origin except for a null vector Such a vector is not found m a 
factor problem because its correlations would all be zero or small so that it 
would be eliminated from the test battery before factoring was begun 
The problem of determining the uniqueness of a simple structure may not 
be so serious as it seems at first sight We are accustomed to the demand of 
statistical methods that a solution shall be unique m older to be classified 
as a preferred method Actually, we do not often attain uniqueness The 
curve-fitting methods in general use are not unique even when the same 
methods are used, as, for example, when a least-squares solution gives sev¬ 
eral variant answeis, depending on which of the variables the investigator 
chooses to be dependent on the rest of the variables However, it would be 
very useful to have a solution to the simple-structure problem by which we 
could know by some explicit method whether or not a frame exists in the 
configuration that gives a better fit than the one we may have found 
In presenting a simple structure as a solution to a factor problem, several 
considerations determine the plausibility and acceptability of the solution 
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The plausibility of a solution should be considered separately for each fac¬ 
tor Some of the relevant criteria will be mentioned bnefly 

1) The degree of ambiguity or plausibility of each hyperplane as judged 
by the configuration In Figure 5 we have illustrated for a three- 
dimensional configuration how the several co-ordinate planes frequently 
differ widely m degree of plausibility and convincingness The plane L\ is 
overdetermined by more than enough points (test vectors) to define the 
plane, and hence the existence of a parameter T\ seems clearly indicated 
The plane La is barely determined, since two points are the least that will 
define the plane in three dimensions The existence of the parameter T 3 is 
suggested but not demonstiated The plane I/ 2 is indeterminate, as shown 
by several possible locations m the dotted lines. The parametei T 2 is sug¬ 
gested but not demonstiated In Figure 6 we have a tinee-dimensional con¬ 
figuration of six test vectors, m which the stiuoture is not at all convincing 




The sets of solid lines and dotted lines show two entirely different simple 
structures, neither of which is ovei'determmed We see here how it is not 
likely that some single cutenon will be found for the uniqueness of a simple 
structure, because the seveial planets and their corresponding parameters 
may differ widely m their degree of overdetermination and convincingness 
in any particular factor study, 

2) The plausibility of the theoretical mteipietation that the investigator 
may be able to find for each hyperplane When an unexpected plane ap¬ 
pears m a factorial study, it is a challenge to find an interpretation for it 
If the interpretation seems plausible and if it clearly divides the tests into 
two categories, those with strong non-vanishing values of V JP and those 
with negligible factor loadings, the interpi etation can be accepted at least 
as a hypothesis worthy of further study This task of interpreting the fac¬ 
tors found is clearly the most important part of a factorial study. If the 
interpretation is foggy, then so is the whole study, no matter how elegant 
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the statistical work may have been Exploratory studies in a new domain 
are likely to give clues only as to the nature of the underlying functions or 
parameters 

3) The predictability of a postulated factor In order to set up a crucial 
test of the word fluency factor W, two new tests of synonyms were con¬ 
structed, one of which was piedicted to have high saturation on W and no 
saturation on the verbal factor V, while the other synonyms test was pre¬ 
dicted to behave in just the opposite way, namely, zero on word fluency 
W and high on the veibal factor V They did behave m the way piedicted 
m the next factonal study This did not prove the intei pretation of W to be 
finally collect, but it did sustain the hypothesis concerning the differentia¬ 
tion of V and W Eventually, the factor W may be redefined by other 
studies so that it might even transcend the verbal content altogether. 
Here, as in other experimentation, success depends very largely on how suc¬ 
cessful we may be m inventing tests and measurements that will be crucial 
in differentiating between rival hypotheses concerning the nature of the 
factors or in testing a tentative interpretation indicated by previous studies 
It may happen that one expcnmentei will find a factor, that another will 
interpret it with some hypothesis, and that a thud will invent the crucial 
tests by which the hypothesis can be examined factonally or otherwise. 
It is in this manner that factor analysis can find an important role as a 
scientific method not only in psychology but m other sciences m which ex¬ 
ploratory studies may reveal an underlying order in a new domain. 

In making interpretations for a new factor that has been identified fac¬ 
torial^, we distinguish, in the spontaneity of our approval, between inter¬ 
pretations that seem vague or far-fetched and those occasional incisive in¬ 
terpretations that “click,” as it were, and by which the factonal results 
snap into a meaningful whole. This is only to say that the scientific mtei- 
pietation of experiments is as subjective now as it always has been To de¬ 
mand that factorial interpietation shall be objective is as absurd as to de¬ 
mand that the interpretation of a physical experiment shall be objective. 
It never is 

4) The degree of convincingness of a simple structure This has been 
described as due, m part, to the overdetermmation of each hyperplane, but 
this judgment is made also with due regard to the relative number of test 
vectors that do not contribute to define the simple structure In Figure 7 
we have drawn two plots that might be found for a pair of columns of a 
factor matrix In the first we could fit the planes anywheie in a random 
configuration with perhaps enough points to determine a hyperplane, but 
it would not be convincing. In the second plot we should feel quite con¬ 
vinced that an underlying order is revealed because the distribution of the 
points makes the location of the two axes quite compelling Here our judg¬ 
ment is influenced by the relative number of points that do not contribute 
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to the definition of the simple stiuctuie In the second plot we have drawn 
seveial points m the middle of the fiist quadiant If an interpretation can 
be found for the factois A and B, then it is necessary to explain how it 
comes about that the seveial tests in the fiist quadrant have some satura¬ 
tion in both A and B If such an explanation seems plausible, it strengthens 
the inteipretation of the factois A configuiation such as the first m Fig¬ 
ure 7 leaves one unconvinced, no mattei wheie the axes are drawn, unless 
an inteipetation can be found that seems light Random configuiations like 
tins seldom yield cleai intei pi etations, but they aie not, of course, physical¬ 
ly impossible 
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Some cutics have pointed out that if the factonal composition is known 
for a large test batteiy, it would be possible to pick and choose a set of 
tests from the larger battery so that they would reveal a simple stiuctuie 
even for some more or less aibitiarily selected leference fiame and that the 
simple stiucture so put together would not demonsti ate any validity foi 
the arbitiary leference frame This is entuely coirect This is only one of 
many ways in which a dishonest author can fake scientific data Anothei 
well-known case is that in which one can pick and choose points in a coire¬ 
lation table, eiasmg the points in the second and fourth quadrants, so as to 
change a zero correlation to a high positive value If a test battery is as¬ 
sembled with tests that are crucial for one or several rival hypotheses, the 
configuiation of the whole batteiy should be used in the intei pretation If 
the interpretation is valid, it should be possible to construct new and diffei- 
ent tests with prediction as to how they should behave factonally Again 
we should want to see the hypotheses, the tests, and the whole eonfigui ation 
Another criticism of simple structure which is in the same class is that 
one gets factonally only wliat one puts into a test battery and hence that 
the separate factors have no validity and repi esent only the arbitrary cate¬ 
gories imposed by an author It is a platitude to say that one gets out of a 
test batteiy only what one puts into it Again, that is true for any scientific 
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experiment. When a test battery is constructed with some hypothesis 
which is to be examined, it does not follow that the factorial analysis neces¬ 
sarily supports the categories that guided the construction of the battery 
Most of our disappointments in factorial studies are, m fact, exactly of this 
kind Hence we can say on the basis of considerable experience that a factor 
analysis does not necessarily reproduce the categories that constitute the 
preconceptions of the investigator Any factors that are found are certainly 
represented by the tests that were put into the battery, but they do not 
necessarily lit the preconceptions of the experimenter. 

The various criteria that have been discussed here seem obvious enough 
when we deal with graphical methods of inspecting the configuration, but 
they are likely to prove troublesome for anyone who attempts to reduce 
them all in some algebraic manner to a single criterion of uniqueness of 
simple structure Any contributions m this direction are hkely to be helpful 
in dealing with partial aspects of the problem of finding the most plausible 
lefeience frame for any given configuration It must not be forgotten that 
it is an open question for each factorial study whether any simple structure 
exists at all or whether any one or several hyperplanes can be defined. It 
should also be noted that a factor study may be a major scientific contribu¬ 
tion if it reveals only a single clear factor which can be given a fruitful and 
provocative interpretation, even if all the other factois m the same study 
prove to be entirely indeterminate Furthermore, such a factorial result 
might be of fundamental significance, even if, say, only a fourth or a thud 
of the variance of the crucial tests is accounted for by the factor in question 
Other things being equal, a factor will be the more convincing, the higher 
the saturations involved in the tests that involve the factor But the exist¬ 
ence of large parts of the variance as unique or as unmterpreted common 
factors does not preclude significant findings 

Factorial studies may be roughly classified as of two kinds When there 
are no hypotheses to guide the study of a new domain, the experimenter 
tries to cover the domain with tests or measures that are diversified withm 
the domain The factorial result is likely to give an incomplete simple struc¬ 
ture at best, but the exploratory study is justified if one or more functions 
or parameters appear in the interpretation that can be used as starting- 
points for moie crucial subsequent studies The second kind is the experi¬ 
ment m which some hypotheses are available and m which tests are specially 
constructed to represent the presence and the absence of each postulated fac- 
toi or parameter Such studies are likely to be more productive of definitive 
results 

Interpretation of zero factor loadings 

There is occasional misunderstanding about the interpretation of zero 
factor loadings A zero factor loading in a test is sometimes spoken of as 
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if it meant the absence of the factor m the test, but that is not the correct 
interpretation A zeio factor loading means that the factor m question 
does not contribute to the variance of the test The difference can be seen by 
an example Consider two hypothetical factors, spelling and writing speed 
If a simple spelling test were given by the time-limit method to a gioup of 
educated adults, the number of right answers might represent nothing but 
speed of writing, because the educated adults could be assumed to be able 
to spell all the simple words Hence the variance of this test would be at¬ 
tributable entirely to writing speed and none to spelling ability But it 
would be mconect, strictly speaking, to say that the test does not involve 
any spelling ability If the subjects were all equally good m a factor or if 
they all reach the ceiling of the test as regards power, the factor would not 
be lesponsible for any individual differences in the test, and the factor 
would not be identified 

If a spelling test of greater difficulty wei e given to the same population 
with ample writing time, then the factorial composition would be markedly 
changed The gi eater part of the variance might then be attributable to 
the spelling factor, while the loading on writing speed would be zero The 
zero loading on writing speed would be caused by the fact that this factor 
contributed nothing to the variance of the test, even though all the sub¬ 
jects actually wrote the test. 

In the interpretation of zero factor loadings we sometimes say that the 
factor is not involved in the test when we really should say that the factor 
is not involved in the test variance Since we deal mostly with common fac¬ 
tors, we speak of the factors m a test when common factors are implied 

The positive manifold 

The principle of simple structuie is frequently described as if it assumed 
all factor loadings to be positive or zero That is peihaps the most common 
misunderstanding about the pi meiple The reason is probably that the fac¬ 
torial methods have been applied more in the field of mental abilities than 
in any other field, and it is here that test corielations have been found uni¬ 
versally to be positive or zero. It is therefore natural to postulate that 
when a unique simple structure is found for a battery of tests of mental 
abilities, then the non-vanishmg entries in the factorial matrix are positive 
But it must be understood that this is a postulate of much psychological 
investigation in a particular field and that it does not constitute a restriction 
upon the principle of simple stiuctuie, which is applicable either with or 
without the special lestriction that the factor loadings shall be positive or 
zero 

The scientific problems to which the factor methods ai e applied may re¬ 
quire different restrictions on the elements of the factorial matrix Some 
of these restrictions may be considered under four cases, as follows: 
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1) The simplest case is that m which the factorial matrix F can be used 
as determined by the eentioid method; or by any other equivalent method, 
without restiictions beyond those that are inherent m F It is piobably 
seldom that a scientific problem can be adequately solved without some re¬ 
strictions on the elements of F. 

2) One form of constraint that is of veiy general scientific interest as re¬ 
gards the factorial matrix is that of simple structure 

3) If the scientific pioblem is such that negative cell entries infaio ex¬ 
cluded, then we have the important case of a simple stiucturc m which 
a }m jg 0 This is the assumption that underlies the application of factorial 
methods to the problem of isolating pnmaiy mental abilities, but the as¬ 
sumption is not absolutely necessary, since ideal constructs can be devised 
for a science of psychology which do not require that the cell entries of F, 
or those of the oblique factorial matrix V, be positive or zero 

4) A special case of the positive entries of F is the further iestnction that 
each factoi of F or of V be either completely present or completely absent 
in each test This is a case of possible interest in genetics, but it is not likely 
that it will be dnectly applicable to scientific data without admitting a spe¬ 
cific vanance for each variable 

If all the elements of V aie positive or zeio, then each column of V is de¬ 
fined by a positive hyperplane so located that all the trait vectois which are 
not contained m it aie on the same side of it If it is assumed that all the 
factors have positive or zeio contributions to each variable, then all the 
trait vectors are in the positive legion The bounding planes of this region 
are then of special mtei est 

It will be convenient to name the bounded space within which any radial 
vector has only positive dnection cosines The bounded space m which any 
radial vector has only positive direction cosines will be called the positive 
region 

If all the trait vectors that do not he in a hyperplane are, on the same side 
of it, the hyperplane will be called a positive hyperplane with reference to the 
trait configuiation 

K, m addition, a set of r positive hyperplanes exists such that each trait 
vector is contained m one oi moie of them, then the combined configura¬ 
tion of the trait vectors and the refeience vectors will be called a positive 
simple structure 

The geometrical interpretation of the restnction upon the numerical val¬ 
ue of v„ in V in the case of mental ability tests is that all the test vectors 
lie in the positive region of the common-factor space When this condition 
is satisfied for orthogonal vectors, all the intertest correlations are positive 
or zero. It is a universally accepted fact that mtertest correlations for men¬ 
tal abilities are positive 

The converse is not necessarily valid The well-known fact that all inter- 
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test correlations are positive implies that all the test vectors lie inside a cone 
with center at the origin and with a generating angle of 7r/4 Such a cone 
cannot be inscribed m the positive region except when the number of di¬ 
mensions is as low as two 

The restriction that all the test vectors shall be in the positive region of 
the common-factor space is not sufficient to determine F uniquely In gen¬ 
eral, there exists an infinite number of orthogonal transformations by which 
all the entries in F become positive or zero if the configuration of F can be 
insciibed in the positive region Special cases may be set up in which one, 
and only one, orthogonal transformation will make the entries a ]m in F posi¬ 
tive or zero Such a case in three dimensions is that in which three test vec¬ 
tors are mutually orthogonal These cases are not likely to be found in 
practice Hence a unique matrix F is not to be expected with the single cri¬ 
terion that a 3 ,„ ^ 0 in F 

The ozthogonal hypeiplanes which bound the positive region m r dimen¬ 
sions will be called the orthogonal positive manifold 

A set of r distinct and oblique positive hyperplanes for a trait configura¬ 
tion in r dimensions will be called an oblique positive manifold 

If the factor matrix of the traits which are contained m a positive hyper¬ 
plane is of lank (r —1), then the hyperplane is a bounding hyperplane or a 
positive co-ordinate hyperplane 

The identification of a positive simple structure is done by noting the 
bounding hyperplancs if they are found in the rotations. If a positive mani¬ 
fold is not postulated or suspected m the test batteiy, the rotations aie 
made toward linear concentrations of test points in the diagrams without 
refeience to the question of whether the hyperplanes are bounding planes. 

Unitary factors 

A special case of the positive manifold is that in which each factoi is 
either completely piesent or entnely absent in each member of the experi¬ 
mental population Each individual member of the population has, then, 
one of only two possible standard scores—one positive, which represents the 
presence of the trait, and the other negative, which represents the complete 
absence of the trait The numerical values of these two possible standard 
scores are determined by the proportion of the population that has the 
tiait If it is assumed m first approximation that the unitary factors are 
equally weighted, then the correlations take only certain possible values 
which are determined by the number of the unitary factors or elements in 
each of the test variables By inspecting the experimental correlations, one 
can make estimates of the number of elements m each test variable This 
method of factor analysis seems to be limited m practical application be¬ 
cause of the serious effects of unique variance and because the elements are 
not likely to be equally weighted m each test variable However, the restnc- 
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tion of the correlation coefficients to certain values makes this principle of 
some theoretical interest, and it might be adapted to factorial studies of cer¬ 
tain types It can be legarded as a different type of restriction on the fac¬ 
torial composition by which the reference axes may be located This method 
has not yet found practical application.* 

Constellations 

In formulating hypotheses concerning the nature of the primary traits, 
it is sometimes a consideiable aid to know of constellations that may exist 
in the trait configuration By a constellation is meant a grouping of trait 
vectors It happens not infiequently that the trait configuration consists 
essentially of groups of trait vectors The angular separations between the 
trait vectois within a constellation axe relatively small, while the separa¬ 
tions between constellations are marked 

When the dimensionality of the factonal matrix is less than four, the con¬ 
stellations may be inspected readily by graphical methods. When the di¬ 
mensionality exceeds thiee, the graphical methods are not available, and it 
is then useful to have a routine by which the constellations may be isolated 
in the trait configuration \ Since the constellations are to be used as an aid 
to intuition regarding the nature of the primary traits, it is not advisable to 
define a constellation rigorously as regards maximum angular separations 
or as legards the maximum generating angle of the cone which will include 
a constellation Such restrictions may be arbitrarily imposed by the in¬ 
vestigator for each study 

If an attempt is made to isolate constellations from a large battery of 
traits—say, fifty or more—without some systematic procedure, it is usually 
found that the groupings become entangled in annoying complexity If the 
constellations do not exist, the procedure must make this fact evident, but, 
on the other hand, constellations can be drawn for the purposes of studying 
the battery, even though the traits arrange themselves more m the nature 
of chains than of constellations In three dimensions this situation is illus¬ 
trated by a battery of traits whose configuration reveals a spherical triangle, 
in which the sides of the triangle are well defined by the trait vectors. If all 
of them lie m the sides of a spherical triangle, then the isolation of constella¬ 
tions would be difficult, because there may be no sharp break between one 
constellation and the next In three dimensions the graphical methods 
would, of course, be used because of their simplicity and directness; but in 
higliei dimensions the groupings may be obtained by inapectional methods 
from the mtercorrelations corrected for uniqueness % 

* Sec first ed , pp 205-12 f See first ed , p 174. 

t Lately, Professor Tryon has proposed that we return to cluster analysis instead of 
the factorial methods that have been developed in the last decade The inspection of 
a correlation matrix for groupings or clusters was naturally the first attempt in analyz- 
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One useful proceduic is to ascertain, first, tlic average con elation in each 
column of the correlational matrix R u , where the given coefficients have been 
corrected for uniqueness (An alternative is to count in each column the 
number of coefficients whose absolute values exceed some specified values ) 
Select the trait T x with highest mean coefficient. List all the traits whose 
correlations with T x exceed the specified value and complete the correla¬ 
tion matrix for the traits so selected Eliminate from the table the trait 
which has the largest number of small mtercorrelations Repeat the elimi¬ 
nating process until all the traits that remain in the table have appreciable 
mtercorrelations These traits constitute a constellation Select the trait 
whose mean coefficient is next highest and which is not listed in the group 
just formed, and proceed with it in the same manner as with T x until the 
majonty of the traits have been assigned These groupings are flexible, and 
they may be arranged to overlap The arrangement of the traits m constel¬ 
lations should be regarded merely as a lough method of mspectional anal¬ 
ysis The procedure here outlined can, of course, be modified in many ways 

A note about static and dynamic factors 

Investigated who are using the factonal methods as an aid in testing 
psychological hypotheses find themselves involved not only in legitimate 
controversies about techniques and about intei pretation with uval hypothe¬ 
ses but also in some forms of cuticism that are not much moie than verbal¬ 
isms One of these that recurs too frequently is the charge that differenti¬ 
able functions or parameters that can be identified factonally are invalid be¬ 
cause they are supposed to be static, mechanical, atomistic, and isolated 
and that any such factors imply behavioral anarchism In contrast, we are 
told that factors, instead of being static, must be functional and dynamic 
“relationships” and that these must somehow embiace the whole personal¬ 
ity It is doubtful whether any real issue exists to justify such talk It is 
doubtful whether any psychologist who has good tiaimng m his subject 
ever really looks for a factor that is truly isolated from the rest of man’s 
mind Further, if a factor is a relation of some land, then surely there must 
be some distinguishable things that are related, and if these things aie not 
to be identified, then the relation will be obscure indeed 

In the box problem one of the factors was the height. That was a param¬ 
eter that entered into the measurements of each box, but it does not fol¬ 
low that the height was m any sense isolated fiom the totality of the box 
But to this example there would be the objection that the boxes and their 
measurements are static, whereas m psychology we must be dynamic In¬ 
stead of using boxes as an example, we could have used flywheels This ex- 

mg the correlations The early mspectional methods for clusters are hardly to be pre¬ 
ferred to the factorial methods, winch identify not only the clusters but also groupings in 
hyperplanes 
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ample would have shown dynamic factois, such as momentum, the radius 
of gyration, and revolutions per second of the flywheels Such an example 
would work just as well in differentiating between factors which would then 
refer to kinetic aspects of the objects Factor analysis would apply as well 
in one case as in the other Pedagogically it is perhaps better to use the 
boxes because the reader can then devote himself to factorial ideas instead 
of trying also at the same time to learn some theoretical mechanics 
Another verbalism in the interpretation of factors is the insistence that 
they must he treated as "relationships ” Let us not talk about relations 
without giving at least some hint about the entities that are being related 
Relations do not float around by themselves They are tied to the concepts 
or things that are related In some current factorial writing, mental organi¬ 
zation is similarly described as consisting of "relationships,” If factors arc 
the relations, then it would be interesting to have some hint about the things 
that aie related 



CHAPTER XV 
OBLIQUE STRUCTURE 


Primary trait vectors 

The fundamental concepts of oblique structuie will be here defined for r 
dimensions Figure 1 has been drawn to show the relations of the reference 
vectors A p , the hyperplanes L p , and the pnmaiy vectois T p Figure 2 has 
been diawn to show these lelations in rectangular foim 

In Figure 2 we have the factor matiix F with elements a ]m , which are de¬ 
termined by the factonng piocess fiom the cori elation matrix The trans¬ 
formation matrix A with elements \ mp cames the factor matiix F into the 
oblique factoi matiix V with elements v JP This is expressed in the equation 

(1) FA = V 


The test vectors j aie defined m F by then projections on a set of oithog- 
onal reference axes The same test vectois aic defined in V by then 
projections on a set of oblique leference axes A p In Figui e 1 the oblique ref¬ 
erence axes A p are shown, but not the oithogonal lefcience fiame Each 
column of the transformation matiix A gives the duection cosines of one 
of the refeience vectors in terms of the oithogonal frame. 

The r liypeiplanes, whose normals pioduce a simple stiuctuie with a test 
configuration, will be called the co-oidinate hyperplanes foi the test configu¬ 
ration The simple sti uctui e is defined by the test configuration and the noi- 
mals A„ to the co-ordinate hyperplanes L v In Figure 1 each refeience vectoi 
A p is orthogonal to the hyperplane L v In particular, the refeience vector 
Ai is orthogonal to the hypeiplane Li, and similaily for the other planes 
The lefeience vector defines the hyperplane, which isasubspacc of (? —1) 
dimensions in the total common-factor space of r dimensions 

The intersection of any set of (r— 1) co-oidinate hypeiplanes defines a 
co-ordinate axis of the structure In Figwe 1 the intersection of the hypei - 
planes Li and L 2 defines the co-ordinate axis T 3 The total number of sets 
of (r —1) hypei planes that can be taken is r, and consequently then mtei- 
sections define r co-oidmatc axes These are of scientific mteiest because 
they define the descnptive categories of the simple ordei m terms of the 
tests 

The test vectois that he m the hyperplane L x aie oithogonal to the lefei¬ 
ence vector Aj, and hence these tests have zeio projection on that lefeience 
vector In Figure 2 these tests are identified by zero projections v ]X in the 
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column 1 of the oblique factor matrix V Each of these projections is deter¬ 
mined by the cross product of a row of F and the first column of A 

The unit vector defined by a co-ordinate axis will be called a pnmai y 
trait vector or a -primary vector The tiait which corresponds to a primary 
vector will be called a primary trait or a primary factor The object of a fac¬ 
torial analysis is to discover the pnmary traits and to describe them in terms 



Figukh 1 


of the tests that are experimentally observed In Figure 1 it is seen that the 
ti ait vector r I\ is the unit vector m the intersection of hyperplanes L 2 and Lz 
The subscript for each piimaiy vector is identical with that of the hyper¬ 
plane which does not contain the primary factor Each primary vector T r 
is the unit vector in the intersection of the (r — 1) hyperplanes L g , where 
qy^p In three dimensions there are three pianos L v which contain the ori¬ 
gin. Their intersections determine the primary vectors T p 
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The mteisection of all the co-ordinate liypcrplanes, excepting L p , defines 
a primary trait vector, which is denoted T p Hence T v defines the linear 
subspace which is common to all the hypciplanes, excepting L p The trait 
vector T p is not contained m the hyperplane L m but it is contained m all the 
other hyperplanes It follows that m Figure 2 the primal y ti ait T p is absent 
in all the tests which have vanishing entiles v JV in column p of V The pri¬ 
mary trait T P is present in all tests that have non-vanishing entries v, v m 
column p of V 



Figure 2 

Since the primary trait vector T„ is not contained m the hyperplane L v , 
it might be inferred that it is identical with the noimal to the hyperplane 
L T This is not necessarily the case If the primary traits are uncorrelated 
in the experimental population, then the vectors T p are orthogonal, and so 
are also the co-ordinate hypcrplanes L p and their noimals A,,. In this case 
the reference vectors A p are identical with the primary vectois T p How¬ 
ever, if the primary traits T p are correlated in the experimental population, 
then the hyperplanes L p aro oblique, and their normals A P are oblique The 
two sets of vectors A P and T v are then, m general, distinct 
The geometrical interpretation of primary traits may be illustrated m 
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three dimensions Let the entries a ]m in each row of F be augmented by the 
multiplier 1/h, The geometnc representation of the augmented co-ordi¬ 
nates is that each test vector is extended to unit length The augmented 
co-ordinates are therefore the direction cosmos of unit test vectors The 
termini of the test vectors can be represented as points on the surface of a 
hypersphere If r = 3, the test configuration can be studied graphically on 
the suiface of a ball 

Let Figure 3 represent the test configui ation, and let the points repiesent 
the termini of the test vectors on the surface of the sphere. Simple structuie' 
is shown by the fact that each point lies in one of the three arcs of great cir¬ 
cles All the tests on the arc 1-2 can be described by two piimary factors, 
since all the corresponding test vectors aie eoplanai The whole set of tests 
can be desenbed by three factors Hence the same primary factoi is absent 



Fiquiu, 3 

in all the tests along 1-2 The subspace 1-2 is defined by the direction co¬ 
sines of the normal to the plane 1—2 Let this normal be denoted A 3 The 
subscript of A 3 refers to the primary trait T 3 , that is absent in the subspace 
Li The vector A 3 is the normal which defines the subspace L s 

By analogy, the vector Ai is the noimal to the plane 2-3, and A 2 is the 
normal to the plane 1—3 If all test vectors in the plane 1—2 represent two 
primary factors and if all test vectors in the plane 1-3 repiesent two piimary 
factors, it is clear that the vector which is determined by the intersection of 
these two planes lepresents the primary factor which the two planes have in 
common, namely, the primary factor 1 In the same manner the other pri¬ 
mary factors, 2 and 3, are determined by mteisections of planes 

Equation (1) is shown m rectangular notation in Figure 2. The equation 
can be written in terms of its elements, 

a,m \/,>p — Vjp 


(2) 
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In Figure 2 let the rows of a matrix T with elements t qm represent a set 
of r pumaiy unit vectois T p The lows of the square matrix T are noimal- 
lzed, since T p are unit vectors These aie the vectors T i, T 2 , and T s of 
Figure 1, and they are defined as intei sections of the planes L P The direc¬ 
tion cosines of the first primary vector, the first row of T, are In, f i2 , f 13 
The dnection cosines of the normal to the plane L- 2 , the second column of A, 
are X12, X 2 2, X32 Since the vector 7\ lies in the plane L 2 , we should have 

(3) UiXu + £ 12 X 22 + 1 13 X 32 = 0 

In a similar manner, since the vector T 1 also lies m the plane L 3 , we have 

( 4 ) £11X13 + £12X23 + £13X33 = 0 . 

A similar substitution of the direction cosines of T 1 in the equation of plane 
Li gives 

(5) £nXn + £ 12 X 21 + £ 13 X 31 = d 1 , 

which should not vanish, since the vector T\ does not he in the plane Li 
Let the constant term be denoted di These three equations aie, in fact, the 
multiplication of the fust row of T and the three columns of A, with piod- 
ucts lecorded in the first row of the diagonal matux I), as shown in Figure 
2 The other rows of T can be regarded in a similar manner, so that we have 
the matux equation 

(6) TA = D , 

where D is a diagonal matrix with diagonal elements d p Hence we get 

(7) T = DAr 1 , 

by which the numerical values of T may be determined The inverse of A 
is normalized by rows to give T 

The matrix T is placed 111 Figure 2 as a continuation of the factor matrix 
F Each row of T can be regarded as if it represented a perfect test j of one 
of the primary factors, each of these tests being of unit communality After 
the transformation A, each of these tests will have unit complexity in the 
oblique factor matrix V, in that there is only one entry d v m each low of 
the corresponding section of V, namely, the diagonal matrix D 

The correlations between the primary factors, R pr „ can be found by the 
fundamental factor theorem, since T is a factor matrix for the pumary 
abilities We then have 


(8) 


R pq = TT ', 
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or, by (7), 

(9) Rpv = LA-'C/IA-')', 
which can be written in the form 

(10) R m = DA -i (A - 1 )'.D , 
and this can be expressed as 

(11) R s „ = D(A'A) _1 Z> 

Equation (11) can be used to find the con elation between the primary fac¬ 
tors directly from A and without writing T explicitly The matrix multipli¬ 
cation (A'A) is first performed, then the inverse is found, namely, (A'A)~> 
The columns and rows are adjusted by premultiplication and postmultiph- 
cation with the same diagonal matnx D to make the diagonal entries of R vq 
unity Let the diagonal entry of (A 1 A) -1 be m p Then the corresponding 
diagonal entry of D is 



Since the deiived pure tests T of the primary factors have unit communal- 
ity, the self-correlations must be unity 

Each diagonal element d p is the scalar product of the primary vector T p 
and the corresponding reference vector A p Since both of these vectors are 
unit vectors, the diagonal element d p is the cosine of the angular separation 
of the primary vector T v and the eoirespondmg reference vector A p We 
then have 

(12a) d p = ^ where q = p , 

m 

which becomes unity in the orthogonal case where T p and A p are identical. 

In case we have an orthogonal simple structure, the reference frame of V 
is also orthogonal Then the transformation A in (1) is an orthogonal trans¬ 
formation, so that A -1 = A' Then in (11) A'A=A~ l A==7 Then, by (11), 
we have R pq — I, as would be expected 

Test vectors as linear combinations of primary vectors 

So far we have considered the test vectors in terms of the fixed orthogonal 
frame (a jm m F) and the pi unary vectors in teims of the same frame, t tm 
m the primary factor matrix T, as shown in Figure 2 Consider again the 
test vectors which he in the co-ordinate planes of Figure 8, and the primary 
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factors 1, 2, and 3 at the corners of the configuration The test vectors m 
the plane 1-2, for example, can be expressed as linear combinations of the 
primary vectors T i and Ti We should then have 

(13) Aj p tp m = a jm f 

P 

m which A JP is the weight given to the primary vector p in the description 
of test vector j This equation can be written in the matrix form 

(14) AT = F , 
and hence the desired weights m A are 

(15) A = FT- 1 , 
but by (7) we have 

(16) A = f(Z)A-‘)-‘, 
which becomes 

(17) A = FAD- 1 . 

By (1) we have 

(18) A = VD-' , 

fiom which we infer that the weights by winch the test vectois j can be ex¬ 
pressed as linear combinations of the primary factors T are proportional by 
columns to the entries in the oblique factor matrix V When the trans¬ 
formation A becomes known by rotational analysis, the proportionality con¬ 
stants of D are the reciprocals of the diagonal entries in equation (12). 

The test correlations in terms of the primary vectors 

Since the test vectors can be expressed as linear combinations of the pri¬ 
mary vectors, the test correlations r ]k can also be expressed in terms of the 
primaries By the fundamental factor theorem for orthogonal factors we 
have 

R jk = FF ', 


(19) 

and by (14) we get 

( 20 ) 


R, k = AT (AT)’ 
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or 

(21) R, k = ATT'A ', 
which, by (8), becomes 

(22) R )k = AR pq A' , 

where R pa is the matrix of eorielations between the primary factors Equa¬ 
tion (22) is Tuckei’s generalization of the fundamental factor theorem to 
the general oblique case * The matrix A is piopoitional by columns to the 
oblique factor matrix V, as in equation (18) If the primaiy factois are un¬ 
correlated, R va becomes the identity matrix, so that 

(23) R lk = AA' , 
oi, by (15), 

(24) R ]k = FT~ l (FT~ 1 )' 

But m this case T is an orthogonal matux, so that T~ 1 = T' Hence the rela¬ 
tion reduces to 

(25) R,,. = FF ', 

which is the fundamental factoi theorem for the case of oithogonal factors 

The equation of an oblique simple structure 

A simple structuie is a set of r oblique hyperplanes, all of which contain 
the origin This set of r hypei planes may be regalded as a degenerate cone 
whose apex is at the origin and whose suifacc consists of the ? hypei planes 
The equation of the hyperplane L„ is 

r 

(26) y r m \ mp = 0 

m-1 


The equation of a simple structure in r dimensions may be written by set¬ 
ting the pioduct of r polynomials, like (26) foi p= 1, 2, 3, , r, equal to 

zero Then we have 


r 

T 


r 


X OT Xm? 


TjjiXjnr 

_ m= 1 

m=-1 


_ m— 1 


* Ledyard R Tucker, “The Role of Correlated Factors in Factor Analysis,” Psycho- 
metrtha, V, No 2 (June, 1940), 142, eq (7) 
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This equation* may be written in the more condensed form 

r 

(28) | | £ 7 »\mj> = 0 

p = 1 Lm=»l 

Fitting (28) to a given test configui ation m which a simple stiucture is 
assumed, we have, for each test j, 

(29) v,iv s % v jr = 0 , 
or 

r 

(30) F[ = 0 . 

p = l 




If the point ] is in at least one of the r hyperplanes of (30), at least one of 
the r factors v ]P vanishes, and hence equation (30) is satisfied. Equation 
(28) or its equivalent, (30), is therefore satisfied by all points in the r co¬ 
ordinate hyperplanes of a simple structure 

In older to determine the best-fitting degenerate cone for a given set of 
n points in a space of r dimensions, equation (28) may be written m the form 
of an observation equation, namely, 


(31) 


n 




= Pi , 


whei e p, is the discrepancy for the point j The best-fitting simple structure 
may be defined as that 111 which 

n 

1-1 


is minimized Hence the cnterion for a best-fitting simple stiucture is the 
minimizing of 


(32) 


sn 

j-i p-i 






= 2 ek = + 


j=i p-i 


The function <f> is then a criterion for the isolation of a simple structure 

* This equation was suggested by Professor Raymond W Barnard, of the Depart¬ 
ment of Mathematics at the University of Chicago 
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Equation (32) is an example of analytical catena foi the isolation of sim¬ 
ple stiuctuie A number of attempts have been made to devise practical 
computing methods with analytical cntena by which the simple-stiucture 
solution would be obtained with fewer giaphs than aie icquued by the 
methods of chapters x and x% Lately, several solutions have been found that 
seem to be computationally feasible and which may replace a pait of the 
graphical work 

The population matrix 

The population matrix P m , foi the common factors is of order rXN with 
elements Each element is the standard scoio v m , of individual % m the 
arbitraiy oithogonal factoi rn In matnx foim we have, foi the test score s 

Jl} 

(33) Sji = > 

in 

or 

(34) A = FP 

By (14) we have 

(35) S = ATP . 

The test scores s Jt can also be expressed as linear combinations of scores 
on the correlated primary factors Then 

(36) = Ajj,Xpi j 

V 

wheie A ]v are coefficients and x vi are standai d scores of individuals i on the 
correlated primary factoi s p In matnx form 11 

(37) jS = AP C , 

wheie x pl are the elements of P c By (35) and (37) we have 

(38) ATP = AP e , 
so that 

(39) TP = P„ , 

wheie P c is the population matrix for the correlated primary factors p 

* Tucker, op cit , p 141 We are using notation A and P for the oblique case, corre¬ 
sponding to Tucker’s generalized notation F and P . 



OBLIQUE STRUCTURE 


357 


The conelations between the primary factors can be expressed in terms 
of oblique population matrix P c Since x mi are standard scores, we have 

(4:0) ^ptl XplXq 1 

% 

or, m matiix form, 

(41) R Pa = ^ PoP' 

This equation can be lelated to the population matnx for the ;u biliary 
orthogonal frame m By (39) we have 

(42) R pq = ~ 0 TP) (TP)' 
or 

(43) R pq = i TPP'T’ , 
which can be wiitten in the form 

(44) R vq = PP'^j r 

But, since the factors in P are orthogonal by definition, we have 

(45) ± PP' = R mM = / , 


and hence 

(46) R P3 = TT' , 

as previously demonstrated 

The oblique observation equation 

The relations of oblique structure have been described here with refer¬ 
ence to the fixed and arbitrary orthogonal frame m The fundamental ob¬ 
servation equation on which multiple-factor theory has been developed is 




-£ 




m 


(33) 



358 MULTIPLE-FACTOR ANALYSIS 

This is the familial equation that we write m matrix form as 


(34) S = FP . 

Here the arbitiary factois of P are those which appear in the factoung of a 
conelation matrix, and they aie orthogonal 

An observation equation can also be written in which the scores s„ aie 
expressed directly m terms of standaid scores on the coi related pnmaiy 
factors. This has been done by Tucker in the form 


(36) 


V 


where .c,„ is the standaid score of individual i in the pnmaiy factor p and 
where a JP are the test coefficients In matrix form we have 


(37) S = AP C 

The relations of oblique stiuctuie can be easily derived fiom (36) as an ob- 
seivation equation foi oblique or correlated factors 
The correlation between tests j and k is 


(47) 


r jk ~ \t 'x* , s iiSki > 

l 


which has been written in the f< 1 m 


(48) R, k = * SS' 

In the same way, the corielation between the pnmary factois p and q can 
be written 


(40) 


A T ^ 1 %p* X qi 


and in matnx form this becomes 


(41) x PJK. 

Substituting (37) in (48), we have 

R, k = ± (AP c )(APaV 


(49) 
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or 



By (41) we have 


(22) VAU', 

which is the generalised factor theorem This becomes 
(25) Bn = W = tr 


when the factors of A ate orthogonal 



CHAPTER XVI 
FACTORIAL INVARIANCE 

Nature of the problem 

The problem of factorial invariance should be analyzed with regard to the 
central purpose of factor analysis, the object of which is to discover a set of 
significant and meaningful parameters for describing a domain In the box 
problem that we have found useful for illustrative purposes, the domain is 
the whole range of measurements which can be taken on rectangular boxes 
and which are functions merely of the shapes of the boxes We have found 
that one simplifying set of parameters for this domain is the lengths of thiee 
adjacent edges, denoted x, y, and z, for the height, the length, and the 
width, respectively Now let one of the measurements s, be determined in 
large part by the height, x, and let it be assumed that m a random collection 
of boxes, selected for analysis, there is marked variance m this parameter, 
the height It would then be expected that the measurement s, would have 
a certain pait of its variance, a® x , attributable to the height factoi in. this 
collection of boxes This is simply the square of the factor loading of the 
factor x m test j If the boxes were to be arranged in order from lowest to 
highest in accordance with the factor x, we should expect to see an array of 
boxes from low to tall Inspection of such an array would aid in guessing 
the meaning of the factor x Ordinarily, one proceeds more simply by ex¬ 
amining all the measures which have high positive, small or negligible, and 
marked negative saturations, m order to guess from the nature of the meas¬ 
urements what is implied by each factor 

As regards the particular measurement s„ which has an appreciable pait 
of its variance attributable to the height factor, its factorial composition is 
evidently dependent on the population of boxes, so that the factor loadings 
would be affected by selection Several collections, such as packing boxes, 
pencil boxes, jewelry boxes, would probably vary in relative dispeisions of 
the three parameters One could imagine a collection of boxes selected so 
that they were all of the same height, The vanance in the height factor 
would then be zero The factor loadings a ]X would vanish, and, in fact, the 
rank would be 2 instead of 3, which is expected for most box populations. 
Hence the factor loadings cannot be expected to be invariant from, one popula¬ 
tion to a different population Any criterion of invariance m factor analysis 
assumes that it is applied to analyses on the same population or to equiva¬ 
lent populations * In psychological analysis this principle means that fac- 


* First ed , p 55 
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tonal composition cannot be expected to be invariant for different age 
gioups, foi example, 01 different groups of subjects, selected by criteria that 
are 1 elated to the factois involved (We shall see later that the configura¬ 
tion may remain mvanant for diffeient populations, but we are here con¬ 
cerned with the numerical invariance of the factor loadings ) Limiting our¬ 
selves here to analyses that are made on the same population, we should ex¬ 
pect to find that if several samples are di awn fiom the same population and 
if independent factor analyses are made with the same battery of tests for 
the seveial samples, then the factor loadings should remain invariant for the 
diffeient samples within sampling errors if the simple structure is complete 
and overdetermmed 

Let the measurement Sj be made on each one of a group of boxes, as well 
as twenty other measurements of the same domain, relating to the rec¬ 
tangular box shapes Now if the measurement s, is heavily saturated with 
the height factor, we should certainly be disappointed with our factonal 
methods if the saturation of on the height factor should turn out to be de¬ 
pendent on the other measurements that are taken on the same boxes. In 
the psychological field there is a comparable situation m studying the satu¬ 
ration of an opposites test, for example, on the verbal factoi If that test 
calls foi a verbal factor, then the satuiation of the factor m the test should 
not change if additional tests are given to the same subjects and then m- 
coiporated into the test batteiy for analysis When the multiple-factor 
methods were being developed, these considelations led to certain demands 
as to what should constitute an acceptable factorial method It is a funda¬ 
mental criterion of a valid method of isolating primary abilities that the weights 
of the primary abilities for a lest must remain invariant when it is moved from 
one test battery to another test battery * The same pnnciple can be stated as a 
fundamental requirement of a successful factorial method, that the factorial 
description of a test must remain invariant when the test is moved from one bat¬ 
tery to another which involves the same common factors f If a test with satura¬ 
tion on a factor were moved to another test battery in which that factor is 
absent, the factor would become a specific factor of the test in the second 
batteiy, and it would not be found by the factonal methods, which are lim¬ 
ited to the common factors This limitation was made in order to make 
the factorial problem determinate Tins requirement is a criterion for the 
acceptance or rejection of factorial methods 

When a correlation matux is factored, the result is a factor matrix with 
a set of orthogonal axes whose location in the test configuration is, in gen- 
eialj aibitrary and entirely dependent on the method of factoring The 
first centroid axis is clearly dependent on the grouping of tests m the battery 
which is being analyzed If theie is a large number of verbal tests m the bat¬ 
tery and only a few number tests and space tests, then it is evident that the 
* Ibid , p 55 f Ibid , p 120 
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fust centioid axis will lie close Lo the verbal tests By changing the propor¬ 
tion of tests of different kinds, the fust centioid axis can be moved about in 
the configuration It is cleai that the projection of a test vectoi on the first 
centroid axis will not be invauant when the test is moved from one battery 
to anothei battery It will depend on the natuie of the othei tests with 
which it is combined in each battery Hence the centioid axis should not 
be used for the interpretation of the factorial composition of any test The 
same is true for all the centioid axes, and the same is also tiue for the pnn- 
cipal axes whose location m the eonfiguiation is dependent on the concen¬ 
tration of tests of different kinds Because this pi meiple should be geometi i- 
cally self-evident, it will not be demonstiated by numerical examples If a 



Figure 1 

numerical demonstiation were to be made, the maximum variation in factor 
loadings would be obtained by having marked vanation in the propoitions 
of different kinds of tests m the two batteries to be compaied 

It was for the solution of this invaiiance pioblem that the simple-stiuc- 
ture principle was introduced In Figui e 1 we have represented two bat¬ 
teries of twelve tests with three common factois, A, B, and C In the first 
battery there aie eight tests near C and only two tests each neai A and B 
In the second batteiy there are eight tests heavily saturated with A and 
only two each near B and C, Let the two batteries have test j in common, 
In each of the triangles we have indicated with an x the appioximate loca¬ 
tion of the first centroid axis or the majoi principal axis It will be seen that 
test 3 would have a low projection on the major principal axis in the first 
battery, while it would have a large projection on the major principal factor 
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in the second battery Test ; is far from the major principal axis in the 
first battery but close to the majoi principal axis m the second batteiy 

If these two test battenes weie factored by any appropnate method and 
if a complete and determinate simple structiue weie found, as bhown in the 
spherical triangles, then we should find m the first battery that test j had 
a high saturation on the factoi A, a small satuiation on the factor B, and 
zei o saturation on the factor C The same 1 esult would be found in the sec¬ 
ond test batteiy, m spite of the fact that the two battenes differ markedly 
m the relative number of tests of each kind The factorial composition of a 
lest in a set of primary factois that have been found m a complete and ovei- 
determined simple structure remains invariant when the test is moved to another 
battery involving the same common factors and m which there are enough tests 
to make the simple structure complete and overdetermmed This is one of the 
most important pimciples in multiple-factoi analysis It could happen that 
when a test is moved fiom one batteiy to another battery, one of the pri¬ 
mary factors would be represented by only two tests Theoretically, this 
would be a common factor, but the location of this doublet factor might be 
indeterminate in the configuration of the new battery For this reason we 
state the principle with the specification that the simple structure should 
be complete and determinate in both of the test batteries before we can have 
assuiance about mvanance of factorial composition If the simple structuie 
is incomplete in one of the batteries, then the locations of the pnmary axes 
are, to that extent, unceitain, and then the faotonal composition of the 
tests would also be unceitain 

Types of factorial invariance 

The simplest form of factoilal invariance concerns the metric invariance 
of the factor matrix solution This is the pioblem of ascertaining the con¬ 
ditions under which the numerical values of the factoi matrix solution are 
invanant. Since theie is infinite variation m the possible locations of the ref- 
eience frame m the configuration, one should not expect any numerical m- 
vaiiance in the projections of the test vectors on the reference axes (the 
factor matrix), unless one imposes some restriction on the location of the 
leference frame It has been shown that the initial reference frame and, con¬ 
sequently, the initial numerical values m the factor matrix are determined 
by the method of factoring which the author is free to choose to suit his con¬ 
venience If any particular method of factoring is specified in sufficient de¬ 
tail, then two authors would generally get the same numerical values in the 
factor matrix, but that sort of computing agieement is not of theoretical or 
scientific significance If we impose the restrictions of simple structure on 
the leference fiame and if such a flame can be found in any given test bat- 
tety, then one can expect mvanance of the numerical values m each row of 
the rotated factor matnx, even when a test is moved from one battery to an- 
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other which also has a deteiinmate simple stiuctuie m the same common 
factors Other restrictions on the refeienee frame which give invariance of 
factonal composition of a tost when it is moved from one battery to another 
can probably be found, but, so far, the simple-structuie principle is the only 
restriction on the reference frame which has given this type of metric in¬ 
variance 

A different type of factonal invaiianoe appears in relation to the selec¬ 
tion of subjects to whom a test batteiy is given This is configurational in¬ 
variance The influence of selection on factonal results can be illustrated 
again with the boxes The first diagram of Figut e 2 may repiesent the result 
of a factorial analysis of a set of measurements on a collection of boxes The 
spherical triangle XYZ repiesents the structure The cosine of the angle 



Figure 2 

between the primary vectors X and Y is the correlation between the pri¬ 
mary factors in the particular collection of boxes measuied A similai inter¬ 
pretation applies to the other two sides of the spherical triangle Now sup¬ 
pose that the same set of measurements was made on another collection of 
boxes Factonal analysis of the mfcercorrelations of measurements on the 
new box collection might give such a structure as is shown in the second 
diagram of Figure 2 Here the con elation between the two primary factois 
X and Y is much higher, as shown by the smaller angular separation between 
the two primary vectors m the second diagram Both of the configurations 
are triangular because three pi unary factois are involved m both popula¬ 
tions The primary factors may be interpreted in the same way foi the two 
populations, but their mtercoi relations ai e different This should not inter¬ 
fere seriously with the identification of each primal y factor, which can he 
done independently for each analysis when the stiuctuie is overdeterrnmed 
It must be remembered that the identity of a factor, the box height, for ex¬ 
ample, is not affected by the fact that it correlates diffciently with other 
factors m different samples of boxes that may be chosen for measurement 
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Confusion about this rather simple idea has been responsible for misinterpre¬ 
tation of factorial results 

In each of the two box collections the measurement j was m both test 
batteries In the first diagram of Figure 2 one infers from the location of the 
measurement j that it is a function of both X and Y and that it does not in¬ 
volve the parameter Z Furthermore, the fact that the test vector j is neaier 
F than X makes it evident that the parameter Y has greater influence on the 
variance of j than does the parameter X The score function s, might be 
some function like (2y+x), which would lie closer to Y than to X because a 
greater part of the total variance of this score function would be attributable 
to F than to X But notice also that the relative location of the measure j 
is the same m the second diagram of Figure 2 This result is caused by the 
fact that, no matter what the correlation between the parameters X and Y 
happens to be in the particular population being analyzed, the score j is a 
function of X and Y but not of the parameter Z. Hence it must lie in the 
side XY of the spherical triangle Furthermore, no matter what the correla¬ 
tion r xv happens to be, the score j is closer to F than to X because of the na¬ 
ture of the score function s, Here we have an example of configurational 
invariance The factor loadings might change maikedly from one popula¬ 
tion to another, but, if the same test battery is used on both populations, 
the configuration should be invariant As a matter of fact, this is a far more 
important consideration m factor analysis than the invariance of the factor 
loadings If the object is to identify a set of significant parameters for the 
description and understanding of a domain, it is more important to be able 
to make the identification as to the nature of the parameters than to ascer¬ 
tain that, in this particular collection of individuals that we happen to be 
studying, the correlations between the parameters have certain numerical 
values and that the saturations of some measure j are such and such for this 
particular group of individuals Later, when the nature of each primary 
factor becomes well understood, it may be a matter of great importance to 
ascertain the numerical values for particular groups of subjects m the form 
of practical norms and the like, but not at the exploratory stage of investi¬ 
gation for which factorial methods have been devised At this stage we 
are groping for some set of concepts or parameters in terms of which the do¬ 
main may be understood. The numerical values should then be regarded as 
being of three kinds, namely, those that are significantly positive, those that 
vanish or nearly vanish, and those that are significantly negative Another 
way of stating the same idea is to point out that it is the factor 'pattern that 
is of importance rather than the factor matrix with its numerical entries 
Configurational invariance can be represented algebraically by expressing 
the test vectors as linear combinations of the primary vectors These rela¬ 
tions will be described in chapter xix, with numerical examples of the re¬ 
sults of univariate and multivariate selection in factor analysis 
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Suppose, now, that a third collection of boxes is chosen for analysis, with 
the same set of measurements, and that all these boxes have the same height 
x The configuration would then collapse to the foim shown m the third 
diagram of Figure 8 The factor -Y disappears because it contributes nothing 
to the vai lances of the measurements, and the rank of the system is 2 in¬ 
stead of 3 The test vector j is then identical with the parameter Y because 
it is a function of V. The contnbution of X is mei ely a constant in the score 
function with no contnbution to the vanance The factor X can hardly be 
expected to be identified when it is completely absent from, the vanances of 
the scores that aie being analyzed Of course, it does not follow that the 
factor X is absent from the individual members of the population It )s 
equally present in all of them and therefore does not contribute to their in¬ 
dividual differences 

Another case of factorial invariance is that in which two diffeient test 
batteries are given to two populations If the simple sti uctui es aie complete 
in both batteries in the same common factois, factorial analyses should give 
the same configurations with different anangements of the test vectors, un¬ 
less some of these aie common to the two batteries The identification of 
the primary factors would be made independently foi the two analyses, and 
it should generally be possible to identify the same primary factors in the 
two batteries, even if they have no tests in common This type of i esult is 
the most convincing form of demonstration of the existence of primary men¬ 
tal abilities and theii interpretation, especially if the faetonal compositions 
of new tests are predicted at the time of their construction and later verified 
with factorial results. It is best in such studies to include seveial tests from 
previous studies as linkage, so as to make the demonstration of the identity 
and nature of the primary factors more convincing It could happen m 
such studies that the new battery would reveal common factors not previ¬ 
ously known The result might be a reinterpretation of the older.primary 
factors or the breakdown of a primary factor into several new primary fac¬ 
tors. These results are dependent on the selection of test batteries, which 
are unlikely to be completely determinate for all the factors, especially in 
exploratory studies * 

One form of factorial invariance that has been legitimately demanded is 
that in which a small test batteiy is selected from a larger one that has been 

* Some time ago the writei was disappointed to find that a certain new test, called 
“Designs,” did not show large saturation on a certain perceptual factor which was the 
object of a factorial analysis Later a possible explanation was found in the discovery of 
a new perceptual factor concerned with speed of closure The old “Designs” test will be 
included in a study with other tests, such as the Gottschaldt Figures, m older to deter¬ 
mine factonally whether this tentative explanation is sustained If that is the case, then 
the interpretation of the perceptual closuie factor can be made with a little more confi¬ 
dence In this manner and as a result of a large amount of experimental woik, we shall 
eventually know something about the mental faoulties 
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shown to have a complete simple structure The analysis of the smaller bat¬ 
tery could be made from a section of the larger correlation matrix In such 
studies it should be expected that the smaller battery would show factorial 
invariance of the factor loadings if the smaller battery were selected so as to 
contain its own complete simple structure. If one of the hyperplanes is un¬ 
stable m either the larger battery or the smaller one, or both, then factorial 
invariance cannot be expected, because there would then be no criterion for 
locating the reference frame which determines the numerical entries in the 
factor matrix 

Rank order and the normalized scale 

In most psychological tests the raw score is merely a numerical index of 
rank order The reason for this interpretation of raw scores in psychological 
and educational tests is that there is no rational origin for the scale, and 
there is no defined unit of measurement which can in any fundamental sense 
be regarded as representing equal increments at the different parts of the 
scale An exception is occasionally found in which all tests in a battery are 
scored in terms of time or of some natural spatial unit in which a rational 
origin can be used. The raw score interval from 20 to 30 points is not regard¬ 
ed as representing in any sense the same increment of ability or achievement 
as the interval from 80 to 90 The origin of the scale is quite arbitrary, being 
determined by the number of easy items, by the scoring formula which is 
more or less arbitrary, and by such arbitrary matters as the time limit. The 
increments at the different parts of a distribution can be modified to repre¬ 
sent large or small increments in ability by the distribution of difficulty of 
the items selected for the test. The shape of a distribution of raw scores can 
be made normal or positively or negatively skewed, at will, by merely al¬ 
tering the time limits and the distribution of difficulty of the individual test 
items Under these circumstances there is little justification for deahng with 
the raw scores as anything more than rank orders for the subjects in an ex¬ 
perimental group 

The problem is sometimes raised as to whether the distribution of intelli¬ 
gence in the general population is normal, but such a question has little 
meaning unless it is modified. One can ask, for example, whether a scale ex¬ 
ists such that the distributions of performances of successive age groups are 
normal on the same scale That is the problem of absolute scaling, and it can 
be rather easily solved. * In dealing with the distribution of raw scores for a 
psychological test, one has the privilege of defining the scale so that a par¬ 
ticular distribution of raw scores is normal, or one can define it so that the 
distribution takes any form one chooses That is a question of definition 
It is not a question of fact or a question to be determined experimentally 

* L L Thurstone, "A Method of Scaling Psychological and Educational Tests,” 
Journal of Educational Psychology, Vol XVI, No 7 (October, 1925) 
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For example, an author has the privilege of defining his measure of stature 
as the logarithm of height m inches if he so chooses The rank orders of his 
subjects remain the same Similaily, one might decide to use the quartile 
range instead of the standard deviation as a measure of dispersion, or the co¬ 
efficient of alienation instead of the correlation coefficient These are exam¬ 
ples of monotomc functions 

If a transformation is made by which the distribution of raw scores be¬ 
comes normal, then one has, m effect, merely altered the numerical values 
of the individual scores so as to retain the same rank order of the subjects 
Such a scale is a monotome function of the original raw scores For any given 
frequency distribution of raw scores one could determine, by empirical 
curve-fitting, the monotomc function which transforms the given raw 
scores to a new set of scores for which the distribution is normal This would 
be an awkward way to solve the problem A simplex way is merely to tabu¬ 
late the percentile ranks of the given raw scores and to assign to each raw 
score the standard score which corresponds to the percentile rank. This is 
determined diiectly from a probability table, such as the Kelley-Wood 
tables The resulting scores are called normalized standard scores or simply 
normalized scores They should not be confused with standard scores deter¬ 
mined by a lineal transformation, which represents merely a translation of 
the origin to the mean and a unit of measurement equal to the standard de¬ 
viation of the distribution This distribution of standard scores retains the 
shape of the original distribution of raw scores, whereas normalized scoies 
give a normal distribution, no matter what the shape of the original dis¬ 
tribution may be In both cases the rank order of the subjects is retained. 

It would be possible to carry out factor analysis with rank orders instead 
of scores, either the raw scores or the normalized scores Such a procedure 
would piobably be discouiaged because, if the underlying distribution is 
assumed to be unimodal of any shape, then the rank orders m the middle 
range should represent smaller increments than the rank orders at the ex¬ 
tremes of the distribution Hence the procedure of using normalized scores 
in factoring is probably more appropriate than to use either rank orders or 
the arbitrary raw scores These considerations make it a desirable practice 
to normalize the raw scores before the factor analysis is made. 

The procedure of normalizing the raw scores before factoring introduces 
a form of invariance All monotomc increasing functions of the raw scores 
retain the same rank order of the subjects Hence all monotomc increasing 
Junctions of the raw scores are factorially identical when the scores are normal¬ 
ised before factoring. Empirical comparisons have been made between sev¬ 
eral methods of treating the original scores. These can be listed as follows: 

1) Factoring the correlations obtained directly from the raw scores 

2) Factoring the correlations of normalized scores 
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3) Factoring the correlations of new scores obtained with a separate ar¬ 
bitrary monotonic transformation for each test and without normal¬ 
izing 

4) Factoring the covariances of the raw scores 

Comparisons of these different ways of treating the scores m factoring have 
shown that if the factor pattern is that of a complete simple structure, the 
different treatments of the scores in the factoring give essentially the same 
result In general, the structure is retained in clearest form by the procedure 
of normalizing the raw scores before factoring, and that is the recommended 
procedure The different treatments of the scores give different correlations 
or covariance entries in the correlation matrix, but the rotated factor ma¬ 
trix and the configuration determine whether the factorial results are the 
same 

Invariance of factor loadings 

A numerical example of the invariance of factor loadings will be described 
here A later chapter will be devoted to the effects of selection on factorial 
results and to configurational invariance. In order to make this numerical 
example as concrete and direct as possible, it was designed as a three-dimen¬ 
sional box problem. Measurements of a random collection of thirty boxes 
were actually made in the Psychometric Laboratory and recorded for this 
numerical example The three dimensions, x, y, and z, were recorded for 
each box. A list of 26 arbitrary score functions was then prepared This is 
shown in the last column of Table 2, which shows, for example, that test 4 
consisted of the area xy of one side of the box 

The data were analyzed in several ways, but only the analysis for the nor¬ 
malized scores will be presented here After the score functions had been 
computed for each of the thirty box shapes, the distributions were normal¬ 
ized so that normalized standard scores were used for the computation of 
correlation coefficients The resulting correlations are shown in Table 1. It 
will be seen m the table of correlations that several pairs of columns are 
identical except for sign This is the result of normalizing. For example, 
columns 15 and 16 in the correlation matrix are identical Test 15 was de¬ 
fined as x/z, while test 16 was defined as z/x The rank orders for these two 
formulae are identical except for direction, and consequently the correla¬ 
tions are identical except for sign In the same way the score functions x and 
x 2 would have been identical m the correlation if x 1 had been included m the 
battery. 

The correlation matrix was factored by the group centroid method, and 
the resulting factor matrix is shown in Table 2 The last column of the factor 
matrix shows the conamunalities. A frequency distribution of the third- 
factor residuals is shown in Table 8, and it is seen that the average of the 
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absolute values of the residuals was 0116, which hardly justifies extracting 
another factor The fact that the three factors were sufficient to account for 
the i ntercorrelations agrees with the fact that three parameters, x, y, and z, 
were used in writing the 26 score functions of this battery 
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The method of extended vectors was applied to the factor matrix of f a 
ble 2, and the lesultmg diagram is shown in Figure 3 The three planes are 
here immediately evident All the points lie on or close to the sides of the 
triangle, except 25 and 26 Referring to the list of score functions in Table 2 
we see that these two test measures were the volume and the mam diagonal 
of the box, i espectively Both of these measures ai e functions of all three of 
the parameters, and consequently the two points 25 and 26 should he inside 
the configuration Hence they do not contribute to the location of the co- 
oidmate planes of the simple stiucture. The list of score functions was so 



designed that most of the scoies were functions of one or two parameters 
and only two of the scores were functions of all three of the parameters 
Hence a simple structure was incorporated in the test battery This is rep¬ 
resented by the fact that all hut two of the scoies defined the triangular con¬ 
figuration of Figure 3 By the method of extended vectors the transforma¬ 
tion A was detei mined, and the factor matrix after rotation is shown in 
Table 2 Diagrams for the two-dimensional sections are shown in Figure If. 

In Table ^ we have the coirelations of the three pi'imary factors, x, y, 
and s, as determined from the battery of 26 measures In the same table we 
have the correlations of the original dimensions of the thirty box shapes 
The two sets of correlations agree closely. 
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When this analysis had been completed, a subbattery of 11 measures was 
selected from the longer list of 26 measures The smaller list of 11 measures 
was made with the restriction that it should be balanced quite differently 
from the larger battery The larger battery was equally balanced as to the 
three primary factors, x, y, and z For the purpose of this demonstration the 
smaller battery was selected so as to contain a relatively large number of 



Figure 4 

tests involving the primary factor y and only a few tests of the other two 
factors The second battery was also selected with the restriction that the 
simple structure should be complete and determinate This selection could 
be easily made from the diagram of Figure 3 The smaller battery was then 
independently analyzed 

The smaller battery is shown in Table 5. The correlations were taken from 
the larger correlation matrix of Table 1. The group centroid method of fac¬ 
toring was used, but any appropriate factorial method could have been 
used, The mean absolute value of the third-factor residuals was 015, which 
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did not justify extraction of a fourth factor The method of extended vec¬ 
tors was used m the centroid matrix of Table 5, and the resulting diagram is 
shown in Figure 5 The corresponding factor matrix after rotation is shown 
in the second part of Table 5 The figure and the rotated factor matrix show 
that a simple structure is present in the smaller battery, as was to be ex¬ 
pected The correlations between the three primary factors determined by 



the smaller battery are shown m Table 6 The several tables can be seen to 
be very similar 

The invariance of the factor loadings can be seen by direct comparison 
of the factor matrices after rotation in Tables 2 and 5. It will be seen that the 
factor loadings for the overlapping tests are very similar. This numerical 
example should serve to illustrate the principle that the factorial composi¬ 
tion of a test is invariant when it is moved from one test battery to another 
battery if the simple structure is determinate in both batteries The in- 
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varancc is independent of the proportions of tests which are heavily sat. 
urated with each primaiy factor, so long as the structure is defined in the 

same common factors 

A distinction must be observed in the discussion of factorial invariance 
between the characteristics of factorial methods and the characteristics of 
the data to be analysed In setting up a criterion for what is to constitute 
an acceptable factorial method, the chief concern is to insure that the meth¬ 
od is adequate to discover what is determinate in the data Factorial meth¬ 
od has not been designed to guarantee the discovery of factors which are in¬ 
determinate in the data. If the purpose of a factorial investigation is to 
study a particular domain or a particular primary factor, then the test bat¬ 
tery should be designed to fit a crucial experiment for the hypotheses to be 
examined The investigator must use his ingenuity m covenng the domain 
so as to make the structure determinate Too often, factorial studies seem 
to be made on correlation matrices which just happen to be available, with¬ 
out reference to the likelihood that they will produce significant results 
Occasionally, such studies do produce interesting findings, but it is safe to 
say that every such exploratory study will indicate the necessity for addi¬ 
tional studies of test batteries specially designed to test the hypotheses that 
may appear in the exploratory study regarding the nature of the underlying 

parameters or factors 



CHAPTER XYII 

ALTERNATIVE METHODS OF ROTATION 

The problem of finding an analytical criterion for the reference frame 

The methods of locating a reference frame for a simple structure that 
have been described have all been graphical, and they have all depended on 
the selection of a subgroup of test vectors for defining each hyperplane The 
graphical solutions m n-dimensional space are generally slow, and they re¬ 
quire a considerable amount of labor Ever since the simple-stiucture solu¬ 
tion was found for the pioblem of locating a reference frame, there have 
been many attempts to arrive at the solution without the graphical proce¬ 
dures Scores of proposed solutions have been tried on a variety of factorial 
problems, but, with a few exceptions, they have not been published None 
of them, so far, has entirely satisfied the desire to find a routine method 
which can be applied to a set of data without calling for any judgment on 
the part of the computer In this chapter we shall describe several pro¬ 
cedures for locating the simple struetuie when it exists in the test battery 
They all require considerably less labor than the graphical methods They 
aie not entirely automatic, in that one cannot do the computations blind¬ 
folded, as it were, but they are m some degree successful, m that a consider¬ 
able portion of the work is done without plotting graphs In fact, only a 
few graphs need be plotted, in comparison with either the method of two- 
dimensional sections or the method of extended vectors, which is really a 
method of taking three-dimensional sections In the final acceptance or re¬ 
jection of a simple-structure solution it is still the appearance of a set of 
graphs that determines the answer 

The problem of finding an analytical solution, for which many attempts 
have been made, can be briefly stated as follows Let it be assumed that the 
correlation matux has been factored so that a factor matrix of r columns is 
available Let it be assumed, further, that a simple structure exists in the 
test battery It would be desirable to be able to write explicitly the direc¬ 
tion cosines of the transformation A in terms of the factor loadings and to 
be able to solve for this transformation directly. 

The methods that have been tiled consist generally of writing some func¬ 
tion 4> of the factor loadings and of the direction cosines of the transforma¬ 
tion matrix A, which is to be maximized or minimized at the desired solu¬ 
tion for each column of A In effect, this general procedure consists in de¬ 
fining a subgroup of test vectors whose projections on one of the reference 
axes are to be minimized. These are the tests which have vanishing projec- 
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ticins in a column of the oblique factor matrix V Since the ideal solution is 
oblique, the solution for each reference vector is independent of the solutions 
for the other reference vectors The choice of a subgroup for defining each 
hyperplane is usually done graphically, but this work can be done, at least 
in part, without giaphical aid 

In practical work there frequently arise various forms of ambiguities that 
cannot be lesolved by statistical methods if the structure is incomplete 
Since it happens not infrequently that the structuie of a test battery is un¬ 
stable in one or more dimensions or else mdeteimmate as to the exact rela¬ 
tion of one or more factors to the test battery, it is best for the author to 
present a set of primary axes as his interpretation of the data Such inter¬ 
pretation can later be revised m the light of further information about the 
factors that appeared unstable in the initial studies In some factorial prob¬ 
lems the structuie is complete and so convincing that it leaves little room 
for doubt m the location of the reference fiame It is for such test batteries 
that it would be useful to have a completely objective analytical procedure 
for locating the reference frame 

A successive approximation method of rotation 

The theory of the method will be given first, aftei which numerical exam¬ 
ples will be given In using this method one starts with a trial reference 
vector that will be denoted A, as in Figure 1 The projection of each test 
vector j on the trial vector A a is computed It is the scalar product /A, = v, q 
These projections constitute, in a sense, a trial column of the oblique factor 
matrix V A frequency distribution is made of the values of v lq . A subgroup 
of tests is selected from this distribution whose projections on a new trial 
reference vector are to be minimized In selecting a subgroup, one may take 
all tests whose projections are in a range of, say, ± 30 or + 20, if a positive 
manifold is not to be imposed on the simple structure If it is assumed that 
a positive simple structure exists, then a subgroup is selected which in¬ 
cludes all negative projections, so that the range chosen would be, say, 
+ .30 to -100 

The first trial vector may be selected by inspection of the correlation ma¬ 
trix One selects a test whose coirelations with the othei tests satisfy two 
criteria, namely, (1) that its range of correlations is lelatively large m com¬ 
parison with those of other tests and (2) that there is an appreciable num¬ 
ber of small projections v jq relative to those of other tests m the battery 
Another good method of selecting trial reference vectors is to choose those 
tests which have highest projections on the centroid axes, although this 
method does not give a good choice with the first centroid axis In Table 1 
we have a correlation matrix of seventeen tests, selected from one of the 
box problems, as an example Inspection of the columns of that correla¬ 
tion matrix indicates that tests 1, 2, and 3 should be suitable trial vectors. 
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We choose one of these and normalize the test vector so that it is of unit 
length This is the first trial vector A g , which is shown in Figure 1. 

In denoting the successive adjustments of a reference vector, we shall call 
its initial position A 0 After the first adjustment it will be denoted A l} after 
the second adjustment A 2 , and so on In the computations for any one ad¬ 
justment, the given position of a reference vector will be called A 0l and the 



Fiqube 1 


new position will be denoted A„, which becomes A„ for the next adjustment, 
and so on until a satisfactory solution has been found 

A correction vector C a is to be found, which is orthogonal to A e , as shown 
in Figure 1 , One such correction vector is found for each of the planes de¬ 
termined by the trial vector A Q and one of the fixed orthogonal axes M of the 
frame, defined by the given factor matrix Hence there will be r correction 
vectors Co to be determined in a common-factor space of r dimensions Let 
the unit vector M in the figure be one of the centroid axes or one of the axes 
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defined by factoring the correlation matrix It is here drawn oblique to A, 
because in general, the trial vector A, is oblique to the centroid axes. The 
projection of A a on M is A„„ which is one of the r direction cosines of A, 
F%aure 1 has been drawn to represent only one correction vector C 0 m the 
plane of A q and one of the fixed orthogonal axes M Then 

A.qM = X m g 

In this discussion, vectors will be denoted by capital letters and scalars 
bv lower-case letteis except for the test vectors which have been discussed 
in other chapters with the notation j Let J be the projection of a test vector 


Table 1 


Correlation Matrix 



1 

2 

3 

4 

a 

a 

7 

8 

1 


24 

26 

75 

76 

26 

90 

55 

2 

24 

35 

77 

37 

79 

59 

92 

3 

26 

35 


38 

80 

82 

37 

40 

4 

75 

77 

38 


67 

67 

95 

93 

5 

76 

37 

80 

67 


69 

77 

58 

6 

26 

79 

82 

67 

69 

54 

. 54 

77 

7 

90 

59 

37 

95 

77 

83 

83 

8 

55 

92 

40 

93 

58 

77 

59 

9 

89 

33 

63 

73 

95 

54 

86 

10 

61 

35 

90 

56 

97 

74 

63 

49 

11 

24 

89 

68 

72 

59 

96 

57 

86 

12 

26 

67 

91 

58 

75 

97 

49 

66 

22 

66 

82 

37 

95 

61 

68 

87 

91 

23 

69 

34 

81 

62 

95 

68 

71 

52 

24 

29 

80 

79 

68 

68 

97 

55 

78 

25 

66 

73 

74 

87 

87 

88 

84 

85 

26 

56 

76 

74 

83 

80 

90 

76 

81 


y 

10 

11 

12 

22 

23 

24 

25 

20 

89 

61 

24 

26 

66 

69 

29 

66 

56 

33 

35 

89 

67 

82 

34 

80 

73 

75 

63 

90 

68 

91 

37 

81 

79 

74 

74 

73 

56 

72 

58 

95 

62 

68 

87 

83 

95 

97 

59 

75 

61 

95 

68 

87 

80 

54 

74 

96 

97 

68 

68 

97 

88 

90 

86 

63 

57 

49 

87 

71 

55 

84 

76 

59 

49 

86 

66 

91 

52 

78 

85 

81 

87 

47 

.58 

65 

89 

54 

82 

73 

87 

61 

83 

52 

95 

73 

84 

80 

47 

61 


90 

73 

58 

94 

85 

86 

58 

83 

90 


60 

74 

93 

86 

86 

65 

52 

73 

60 


56 

70 

85 

86 

89 

95 

58 

74 

56 


68 

84 

81 

54 

73 

94 

93 

70 

68 

87 

.87 

91 

82 

84 

85 

86 

85 

84 

96 

96 

73 

80 

80 

86 

86 

81 

91 



The projection of j on M 


1 on the plane of A, and M, the plane of the figure, 
is a ]m , which is a cell entry m the given factor matrix F Then, since M is a 

unit vector, 


^ 2 ^ MLJ — Ojjm • 

The projection of j on A, is v M , one of the piojections to be minimized by an 
adjustment in the position of the reference vector Then, since A a is also a 
unit vector, 


(3) JA Q - v, q 

Applying the orthogonal correction vector Co to A, in the plane of A, 
and M, we have the long reference vector Al, namely, 

Aq + Co = Ai 


(4) 
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The direction of A l can also be defined conveniently by a correction vector, 
C m , which is collinear with M. * Let 

( 5 ) C m = -*sM . 

Then 

(6) A, = A a -f C m , 
where A, is collinear with A/,. We can write 

(7) A r = <Aj, 

where A r is a unit reference vector and t is a stretching factor 

The projection of the test vector j on the unit reference vector A r is v !r . 
It is the scalar product 

(8) J A r = Vjr * 

The length of the correction vector C n is to be determined so that the pro¬ 
jections v ]T of the tests m the selected subgroup are minimized Substituting 
(7) in (8), we obtain 

(9) tJ A, — Vjr , 
and, by (6), 

(10) t/(A, + C m ) = v, T . 

By (5) 

(11) tj(A q — sM) = Vjr , 
which becomes 

(12) tJA Q — tsJM = Vjr. 

By (2) and (3), 

(13) tVjq tSQ/jr ,, — Vjr 

The problem now is to find the length s of the vector Cm so as to minimize 
the projections v, r Squaring both sides of (13), we get 

(14) tfvjq — 2t‘ 2 sv ig a jm + { 2 s 2 o“m = v] r 

* The vector A l is here defined with only one correction vector, Co It will later be 
defined with r such correction vectors, one for each centroid axiB M. 
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Summing for all tests m the selected subgroup, we have 


(15) — 2t 2 s'y^v ]l ,a lrn + iVVa ! , = y v> T ~ u 

13 11 

Writing the partial derivative of u with respect to s and setting it equal to 
zero, we obtain 

(16) -2< 5 V; v Jq a, m + 2 Ps V a* m = 0 , 

3 

from which 


(17) 


V ]q (X jm 

_ I _ 


The values of a ]m are known in the given factoi matnx F, and the values of 
v, Q are known, since they are the pi ejections of the test vectors in the Bub- 
group on the first trial reference vector A, Hence the length s of the correc¬ 
tion vector C m is known and also the direction of the vector A r 

If the given projections v jq and a )m arc plotted on rectangular cross-sec¬ 
tion paper with the axes A, and M, drawn at right angles, the termini of the 
test vectors j will be represented by points On such a graph the location of 
A r can be drawn so as to minimize the projections v ]r The best-fitting line 
through the points, by the method of least squares, will be orthogonal to A r 
through the origin The slope of the best-fitting line is s, which is also the 
length of the vectoi C m , by which the projections of the test vectors in the 
subgroup on the vector A r are minimized 

With the r correction vectois C m known, we proceed to add them to A, 
and thereby determine a vectoi to be denoted A u , which, when normalized, 
becomes the new unit reference vector A P The vector A L is defined by the 
equation 

(18) Ax, = 7tA„, 

where 7c is a stretching factor by which kA, has unit piojection on A,. By 
(5) and (6) we have 

(19) 7c(A, - sM ) = Ax. 

Since Ax. has unit projection on A„ and since A, is a unit vectoi, we have 

(20) Ax.A, = 1 , 
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or, by (19) and (20), 

( 21 ) k( A, - sM) A„ = 1 , 

which becomes 


(22) kAl — ksM A, = 1 . 

By (1), and since = 1, we get 


(23) 

so that 

(24) 


fc ksAjftq — 1 , 


k = 


1 


1 - sX 


mq 


One of the direction numbers of A l is represented m Figure 1 by the pro¬ 
jection of A L on M This is A mt The value of X m( can be written 

(25) A lM = X m! . 

By (19) and (25) we have 

(26) fc(A, - sM)M = X m! , 


or 


(27) kA q M — ksM 1 = X m! , 
and by (1) this becomes 

(28) k(A mq s) — X OT 1 


Substituting (24) m (28), we get 


(29) 


Xm® S 
1 SXmg 


which determines the direction numbers X m ; Equation (29) is evaluated, 
separately for each of the r direction numbers X m i of A/,. The direction co¬ 
sines \ mq of the trial reference vector A s are known, and the vector length s 
of C m is determined by (17) for each of the r columns of the factor matrix F. 

When the direction numbers A m i of Ax, have been computed, this vector 
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can be normalized to unit length, and it can then be used as a new trial 
reference vector A ( Its direction cosines are then given by 

(30) X., = 

It has been' found that this method of locating the co-ordinate hyper¬ 
planes is more rapidly convergent if the vector A ( is adjusted so as to be 
orthogonal to the centroid vector A„ of the subgroup s The adjustment can 
be made on the vector A t in the plane of Ai and A„ so that its adjusted posi¬ 
tion A u is orthogonal to A„ The adjustment vector may be denoted B, so 
that 

(31) A, + E = A„ ; 
and, since A u is orthogonal to A„, we have 

(32) A„A„ = 0 . 

By (31) we have 

(33) (A, + E) A„ = 0 

The adjustment vector E can be written as a fraction of A c , so that 

(34) E ~ 6A C , 
where b is a scalar Then 

(35) (A ( + bA 0 )A c = 0 , 

or 

(36) A(A„ + 6A1 = 0 , 
so that 

(37) b = - ^ . 

A l 

Then, with h known, we can write, from (31), (34), and (35), 

(38) A u = A ( — Ac , 



which defines the vector A, 
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Equation (38) can be written m terms of the r components of A„. We 
then have 


(39) 


Xml 


£ 

m _ 


Xmc t 


where X mu are the direction numbers of A„, and X mc are the direction num¬ 
bers of the centroid vector A 0 The components of Ac can be determined 
from simple summation in the factor matrix F over the subgroup s We 
then have 

(40) X me , 

* 

which may be denoted briefly a on the computing forms 

a 

The scalar product A ( A 0 can be written in summational notation as 


(41) 


A (Ac = 


^ ^ X m (X 


me j 


and the denominator A? can be similarly written as 
(42) A? = ' 

m m \ a / 


If we let 


(43) 


b 


A (Ac 

K ’ 


we can write the computing formula as 


(44) Xmu Xml b\me 

When the vector A u has been determined, it is normalized and then becomes 
the new reference vector A p . If this unit reference vector is to be adjusted 
again, it becomes the starting-point, A„ for the next trial 

This method is simple as to the numerical work, which can be easily done 
by a clerk without instruction in factorial theory The computational work 
will now be illustrated by numerical examples, 
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Numerical example No. 1. 

For the numerical example of this method of locating the co-ordinate 
hyperplanes of a simple structure we shall use a seventeen-variable box 
problem selected from the numerical example of the previous chapter This 
battery represents a three-factor positive structure The correlation coeffi¬ 
cients are shown in Table 1 A corresponding factor matrix F is shown in 
Table 2 The computations are shown here without the summational checks 
which are essential in practical work. 

In Table 3 we have the same factor matrix recorded with only the first 
digit. It is used in some of the cross products in order to simplify the com¬ 



Table 2 

Factor Matrix F 

Table 3 

Factor Matrix F, 
(One Decimal ) 



I 

II 

III 


X 

II 

III 

1 

65 

~ 67 

.33 

1 

6 

- 6 


3 

2 

74 

53 

37 

2 

7 

5 


3 

3 

75 

00 

- 64 

3 

7 

0 

— 

6 

4 

87 

- 04 

48 

4 

8 

0 


4 

5 

88 

- 40 

- 24 

5 

8 

- 4 

— 

2 

6 

89 

41 

- 20 

6 

8 

4 

— 

2 

7 

84 

- 35 

43 

7 

8 

- 3 


.4 

8 

86 

22 

43 

8 

8 

2 


.4 

9 

83 

- 56 

- 03 

9 

8 

- 5 


.0 

to 

85 

- 26 

- 44 

10 

8 

- 2 

— 

4 

11 

86 

49 

- 01 

11 

8 

4 


.0 

12 

87 

29 

- 38 

12 

8 

2 


3 

22 

86 

05 

47 

22 

.8 

0 


4 

23 

86 

- 34 

- 32 

23 

8 

- 3 

— 

3 

24 

89 

39 

-.16 

24 

.8 

3 

— 

1 

25 

99 

- 01 

01 

25 

9 

.0 


0 

20 

96 

10 

- 02 

26 

9 

1 


0 


putations, since one significant figure for the weights is sufficient for the 
purposes of this method. 

When the factor matrices have been written as m Tables 2 and 3, a data 
sheet is prepared as shown in Table 4 The first step is to select as many in¬ 
dependent test vectors as there are common factors to be normalized and 
used as the initial trial reference vectors These trial vectors are to be ad¬ 
justed by successive approximation until we find a set of reference vectors 
that defines a simple structure if it exists in the data 

Tests 1, 2, and 3 satisfy the criteria of having a wide range of correlation 
coefficients in their respective columns of the correlation matrix and also 
an appreciable number of relatively low coefficients These three test vec¬ 
tors will be used as starting-points for the computations The first section 
of Table 4 shows the factor loadings of these three tests, copied from Table B 
These three test vectors are normalized, and the resulting direction cosines 
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are shown in the columns A a , A b ,A c of the transfoimation mat.ix A 0 These 
three unit vectors are the normals which define the three trial planes, A, B, 
and C 

In the matrix Vo of Table 4 we have the pi ejections of the normal-length 
test vectors on these thiee reference axes These are obtained by the com¬ 
putations FAa = Vo In the same table we also have three frequency dis¬ 
tributions of these projections, one for each of the three trial vectors The 

Table 4 


Example 1 Initial Vectors 


Trial Test V ectors 



Matrix Ao 



c= 

A'A 



, Test 1 

Test 2 Test 3 


Aa 

As 

Ac 


A A 

A B 

Ac 

1 

65 

74 

75 

I 

66 

75 

76 

A A 

I 01 

25 

25 

II 

- 67 

53 

06 

II 

- 68 

54 

06 

A, 

25 

1 00 

36 

III 

33 

37 - 

64 

III 

33 

38 

- 65 

A a 

.25 

36 

1 00 


Matrix Vo Distribution Do 



A 

B 

C 


A 

B 

C 

I 

99 

25 

24 

9 

3 

3 

3 

2 

25 

98 

35 

8 


4 

4 

3 

24 

35 

99 

7 

2 

2 

2 

4 

76 

81 

35 

6 

3 

2 

2 

5 

77 

35 

80 

5 

3 



6 

24 

81 

83 

4 


— 


7 

93 

60 

34 

3 


• 5 

5 

8 

56 

93 

39 

2 

6 

1 

1 

9 

91 

31 

62 

1 




10 

59 

33 

92 

+ 0 




n 

23 

91 

69 

- 0 




12 

25 

66 

93 





22 

68 

84 

34 





23 

69 

34 

84 





24 

27 

82 

80 





25 

66 

74 

75 





26 

56 

77 

75 






lowest six test vectors are designated by then piojections in each distribu¬ 
tion as constituting a subgroup s Each of the three subgroups is to be used 
for defining a plane 

Finally, we record the matrix C, which shows the cosines of the angles be¬ 
tween the three trial reference axes Inspection of this matrix shows that 
no two of them are sufficiently close together to define the same plane 

When the computations of Table 4 have been completed, a data sheet is 
prepared like Table 5 The top section of this table shows the computations 
for the adjustment of the A-plane, and it will be described m some detail 
A strip is prepared for the six tests in the subgroup for plane A. In the dis- 
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tribution of Table 4 we see that the subgroup for the A -plane contains all 
projections below 30. In. column A of the matrix F 0 we find that these six 
teste are 2, 3, 6, 11, 12, and 24 The factor loadings of these six tests are 
copied from column A of Fa, and they are recorded near the edge of the 
strip in the proper position to match the factor matrix of Table 3 The 
cross products, Eva, are then computed. In doing this, one places the strip 


Table 5 


Example 1. First Trial 
A-Plane 



1 

2 

3 

4 

6 

6 

7 

8 

0 

- 

Zva 

Xa* 


• (1 

— sAfrtfl) 

^•17*1 

^tnfl 

Xmu 

Imp 

I 

II 

III 

1 135 
444 
- 219 

3 64 
70 
69 

66 

- 68 
33 

32 
03 
- 37 

79 

1 43 

1 12 

43 
~ 92 
63 

6 OQ 
2 17 
— 1 02 

51 
- 89 
01 

43 
- 75 
51 


E-Plane 


1 469 3 41 

75 

43 

68 

47 

4 

82 

69 

40 

Gl3 90 

54 

- 68 

1 37 

89 

— 2 

16 

84 

65 

439 74 

38 

- 59 

1 22 

80 

-1 

34 

77 

60 


C-Plane 


45 

.66 

47 

4 81 

50 

46 

01 

1 00 

06 

- 20 

05 

05 

91 

1 59 

- 98 

2 51 - 

96 - 

89 


I 

II 

ill 


I 585 3 41 70 

,007 74 . 00 

.745 82 - 05 


Coefficient b 



A 

B 

G 


— 4890 

— 7200 

- 2121 


30 7493 

20 6930 

20 5038 

b 

- 0159 

- 0246 

- 0072 


A, 



A 

B 

c 

i 

43 

40 

46 

rr i 

~ 75 

65 

05 

hi 

51 

,60 

- 89 


A’Ai = C 



! A 

J3 

a 

A 

1 01 

02 - 

29 

B 

I 02 

99 - 

29 

C 

- 29 

- 2D 

1 01 


Matrix V, 



A 

B 

G 

1 

95 

00 

— 03 

2 

11 

91 

04 

3 

i - 05 

00 

92 

4 

65 

66 

- 03 

6 

56 

.00 

60 

6 

- 03 

66 

61 

7 

84 

42 

- .01 

8 

42 

80 

02 

0 

75 

01 

38 

10 

34 

- 04 

77 

11 

- 00 

71 

.43 

12 

- 04 

as 

76 

22 

57 
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adjacent to columns I, II, and III of Table 3 The cross pioducts are re¬ 
corded m column 1 of Table 5. 

In the same manner the sum of the squares of the six factois loading oi 
each column of Table 3 are computed and recorded m the second column of 
Table 6. 

The values of s are determined by equation (17) This is, computation¬ 
ally, merely dividing the entry of the first column by that of the second 
column. The ratio s is recorded in column 4 
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In column 3 are recorded the direction cosines of the trial reference vector 
for the A-plane from the first column of A 0 

Column 5 is self-explanatory The entries of column 6 are determined by 
equation (29), and they are computed from columns 3, 4, and 5 

Column 7 is computed by equation (40) These entries are merely the 
sums of the factor loadings in the three columns of Table 2, the summation 
being taken only for the subgroup 

The entries of column 8 are determined by equation (39) The coefficient 
b is computed as shown m the fourth section of Table 6 Its value for the 
A-plane is — 0159 as shown 

The entries in column 8 are normalized to give the direction cosines of 
column 9 These values are also recorded m the new transformation matrix, 
which is denoted Ai in Table 6 Matrix C is also computed as shown, and 
it is seen that the three reference vectors are independent, since none of the 
side entries approach unity In higher dimensions this fact is not a guaran¬ 
tee of linear independence, but it is usually a safe assumption 

The same computational procedure is followed "or the adjustment in the 
normals for the B-plane and the C'-plane, as shown m the corresponding 
sections of Table 5 The given factor matrix of Table 2 is now multiplied by 
the new transformation matrix, A, of Table 5, and the result is the new 
oblique factor matrix V i of Table 5. The frequency distribution of factor 
loadings in each of the three columns is shown in Table 5, and a subgroup is 
indicated in each distribution It is a matter of judgment how large to take 
a subgroup If it is taken too large, the corresponding factor loadings cannot 
be made vanishingly small If it is taken too small, an indeterminate solu¬ 
tion may be obtained which is revealed in the final graphs In the present 
case the lowest six tests m each distribution were taken to constitute the 
subgroup for the next adjustment It is, of course, not at all necessary to 
have the same number of tests in each subgroup The size of the subgroup 
usually varies from one plane to another, and it depends entirely on the 
factorial compositions of the several tests in the battery 

In Table 6 we have the same type of computations for the next adjust¬ 
ment of the reference vectors, and it is seen in the frequency distributions 
of this table that the reference axes have been so adjusted that all the tests 
in each subgroup have projections that are close to zero for their respective 
reference axes 

The next step is to plot the three graphs AB, AC, and BC These are 
shown in Figure 2, and it is seen that the positive simple structure is here 
evident and is complete in all three of the common-factor dimensions These 
graphs are plotted from pairs of columns of the matrix Vi of Table 6. One 
more adjustment of the reference vectors would bring the configuration 
even closer to perfection, but the appearance of the three graphs hardly 
justifies any additional refinement. 
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Numerical example No 2 

The first numerical example started with three trial vectors that consti¬ 
tuted a good choice so that the simple structure was readily identified with¬ 
out graphical assistance except for the final set of three graphs in Ftgure 2, 
by which the structure was accepted as complete in all three of the dimen¬ 
sions. In the present example we shall start with a bad choice for the third 
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trial vector C in order to illustrate how to alter the subgroups with graphical 
aids so as to arrive at the desired planes with a minimum of computation 
It will be seen that, even with a bad initial choice of trial vector, the desired 
solution can be found. 

Instead of starting with the test vectors 1, 2, and 3, as in the p evious ex¬ 
ample, we start here with the test vectors 1, 2, and 5. At the top of Table 7 
the factor loadings of these three vectors are listed, and in the adjacent 
matrix A 0 we have the same three vectors normnluwl in nnna+T+nfo fVio ini- 
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tial reference vectors A a , A b , A c . In the next matrix, C, we have the cosines 
of the angles between the three trial vectors Here we notice at once that 
the cosine of the angle between A and C is 77, which is unusually high, In 
the matrix Fo we have the projections of the test vectors on the three trial 
reference vectors, and we also have in Table 7 a frequency distribution of 
the piojections. In columns A and B the lowest six tests were chosen for 
subgroups, and in column C the lowest seven tests were chosen as a sub- 
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Example 8. Initial Vectors 
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The computations of this example were made without plotting pairs 
of columns of F 0 , but in practice this would be preferable! In Figure 8 we 
have two of these plots, namely, AB and AC. In these plots the subgroups 
A and B have been circled, and they agree with the subgroups chosen m 
the frequency distribution of Table 7. In the diagram AC we would choose 
the subgroup that has been circled It is not the same as the subgroup 
chosen in the frequency distribution for C in Table 7 If the subgroup had 
been chosen from the plot of Figure 8, then the solution would have been 
obtained m the present example in one less trial Ignoring Figure 8, we take 
the subgroups marked m the frequency distributions of Table 7 
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carried, out according to the plan described for the first example The result 
was a new set of trial reference vectors, as shown in the matrix Ai, whose an¬ 
gular separations are shown by the cosines m the matrix C. The projections 
of the test vectors on the new trial reference vectors are shown in the ob¬ 
lique factor matrix V u and the frequency distributions of these projections 
are also tabulated At this point giaphs are made, and we show in Figure 4 
the graph foi columns A and C of TL The subgroup A is the same as in the 
frequency distribution of Table 8, but the subgroup C, which is circled m 
the graph, is not indicated by the distribution A new subgroup C is chosen 
from Figure 4 It consists of the tests 2, 8, 22, 4 > 7> and 1. With these sub- 



Fiqtjre i 


groups we proceed to the second trial with computations as shown in 
Table 9 

The result of the second trial computations of Table 9 gives a new set 
of trial vectors in the columns of matrix A 2 , with angular separation as 
shown by their cosines in matrix C The projections of the test vectors on 
these trial vectors are shown in the oblique factor matrix Vi, below which 
we have the frequency distributions of the projections 

A plot of columns A and C of Vi is shown in Figure 5, where the subgroup 
G has been circled It is the same subgroup as determined in the previous 
trial of Figure 4, and hence the same subgroup C is retained for the third 
trial with the computations of Table 10 Note that the s values are all small, 
except for the reference vector C, which has an appreciable adjustment in 
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fchis trial. A plot of the pairs of columns of V 3 in Table 10 shows the same 
structure as in Figure 2, which was accepted as final in the first numerical 
example Note also that the matrix A 3 of Table 10 is practically the same as 
the matrix A 2 of Table 6, which terminated the first numerical example 
This second example has been arranged primarily to show how the sub¬ 
group can be alteied by graphs, which may be plotted for each trial This 


Table 9 


Example 8 Second Tnal 
A-Plane 



1 

2 

3 

4 

5 

6 

7 

8 

0 


Xm 

2a a 

^no 

3 a 

— *^Tn<?) ^ml 

< 

^mu 

j 


- 006 

3 64 

43 - 

- 00 

1 00 

43 

5 00 

42 

44 

LI 

038 

70 - 

* 75 

05 

1 04 - 

- 77 

2 17 - 

- 77 - 

- 80 

III 

080 

59 

51 

14 

Oi 

40 

-1 02 

40 

41 


B -Plane 

I 

- 004 

3 41 

40 ■ 

- 00 

1 00 

46 

4 82 

46 

46 

II 

- 027 

00 

65 

- 03 

1 02 

67 

-2 16 

67 

67 

irr 

040 

74 

00 

05 

07 

57 

-1 34 

57 

57 


C-Plane 


291 

3 41 

38 

00 

97 

30 

4 

81 

37 

46 

560 

74 

- 68 

- 70 

48 

17 

— 

26 

17 

21 

161 

82 

- 62 

20 

1 12 

- 73 

2 

51 

- 69 

- 86 


Coefficient b 



A 

B 

C 

-ZXm/X'n, 

0711 

0002 

~ 4335 


30 7493 

29 6936 

29 5038 

b 

0023 

0002 

- 0147 


A2 Ai A2 — C 



A 

B 

C 


A 

B 

C 

T 

44 

40 

46 

A 

1 00 

- 10 

- 32 

IT 

- 80 

67 

21 

B 

- 10 

90 

- 14 

m | 

41 

67 

- 86 

C 

- 32 

- 14 

1 00 


Matrix Vs 



A 

B 

C 

1 

96 

04 

- 13 

2 

05 

91 

13 

3 

02 

02 

91 

4 

61 

65 

- 02 

5 

61 

00 

63 

6 

- 02 

57 

67 

7 

83 

40 

- 06 

8 

38 

79 

07 

9 

79 

- 00 

29 

10 

40 

- 03 

71 

11 

- 02 

72 

61 

12 

- 01 

38 

79 

22 

53 

69 

- 00 

23 

52 

- 01 

60 

24 

01 

58 

63 

25 

45 

45 

44 

26 

33 

60 

48 


Distribution D* 


ABC 


9 111 

8 1 

7 12 2 

6 2 2 3 

5 2 3 2 

4 2 2 2 

3 2 1 

2 1 


method of successive approximation can be used for factorial problems m 
which the assumption of a positive manifold is not imposed. The principal 
reason why the computational labor is reduced is that each trial vector is 
adjusted in all the dimensions of the common-factor space m each trial in¬ 
stead of being adjusted in only a few dimensions from the graphs in each 
trial Hence the total number of trials is reduced The successful selection 
of the subgroup of test vectors which is to determine each hyperplane is 
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best made by graphical inspection. It must also be remembered that the 
final acceptance of a simple-structure solution for a test battery depends on 
the appearance of the graphs The various criteria by which a set of graphs 
are interpreted have been discussed in a previous chapter on configurations 
and factor patterns 

The method of minimizing weighted sums 

Theory of the method 

One of the simplest methods of solving the rotational problem m multiple- 
factor analysis is that of locating each reference vector so that the absolute 



sum of test projections is a minimum for those tests that lie m or near the 
hyperplane that is defined by the reference vector If that group of tests 
were known beforehand, then the rotational problem would be easily solved, 
but it will be shown that the desired solution can also be reached with suc¬ 
cessive approximation, 

Suppose that a unit trial reference vector A, is located quite arbitrarily in 
the common-factor space and that the projections v jq are computed It is 
desired to move this trial vector A a in successive steps to the position of a 
primary reference vector Here we shall differentiate between two assump¬ 
tions that the investigator can make, namely, that the nearest primary ref¬ 
erence axis is bipolar or that it is positive The computational outcome is 
not affected by a wrong assumption as to whether the nearest primary refer- 
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ence vector is bipolar or positive, but the solution is reached more quickly 
if the assumption is correct in this regard If the original correlations are all 
positive or zero, it is a fairly safe assumption that the structure is a positive 
manifold. If the original correlations are both positive and negative, then 
some of the primary reference vectors may be positive while some are bi¬ 
polar, or they might all be bipolar 
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Let it be assumed that the projections v }q cover a wide range of both posi¬ 
tive and negative values If it is assumed that the nearest primary reference 
vector is positive, then we should want to move the trial reference vector so 
that, in general, the negative projections will rise toward zero, so that the 
small positive projections will fall toward zero, and so that the high positive 
projections will remain high, or rise still higher, to identify the factor This 
adjustment would be accomplished by moving the trial vector A 5 so as to 
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increase the algebiaic sum of the projections except foi the small positive 
projections which should be decieased 

Let it be assumed, next, that the neaiest primary leference vectoi is bi¬ 
polar Then we should want to move A, so that the strong positive projec¬ 
tions rise and the stiong negative projections fall, thus identifying the tests 
with high variance on the factor The small projections should move toward 
zero Here we want A, to move so as to let all projections rise except the 
small positive and the stiong negative ones, which should decrease algebrai¬ 
cally 

If it is desired to locate a residual plane, then the trial reference vector A, 
should be adjusted so that all its projections approach zero Then all posi¬ 
tive projections should fall, and all negative projections should rise 

These three cases can be summarized for convenience m Table 11 The 
positive and negative signs in this table indicate for each case whether the 
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Projections 
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+ 

4* 
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+ 
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4- 


adjustment of the trial vectot should be made so as to make the projections 
v }g rise oi fall, 1 e , to increase oi deciease algebraically 

In Figure 6 we have aibitrary trial voctoi A, of unit length and with di¬ 
rection cosines \ mQ The subscript m lefeis to the axis of the fixed orthogonal 
leference frame The test vector J has the direction numbers a Jm , so that (he 
projection v lq then becomes 


(45) 



The algebiaic sum of the projections of all the n tests on A„ then is 
(40) Vj , U,„, . 

J in j 

Let one of the test vectois be denoted J, as in the figuie, with projection 
v ia on A r „ as shown Let K be anotliei test vectoi with negative projection 
r in on A a , as shown in the figuie If it is desned to move A,, so that both these 
piojections appioach zero, then it is necessary to increase v Lg and to de¬ 
crease v ]Q The same object can be attained by dealing with J and —Am 
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which the direction numbers a km have been reveised Then the desned ad¬ 
justment of A a is such as to decrease both v )q and v h] Let us denote the vec- 
tois whose piojections ate to be increased or decreased by w,J, where w, 
takes the values +1 or —1, depending on whether the test vector is re¬ 
versed 

Further, let <t> denote the algebraic sum of projections of all the test vec¬ 
tors WjJ on A, This sum is the cntenon which is to be maximized or mini¬ 
mized as the case may be Then we have for this criterion 

(47) 4>p ~ y ) X mt y w,a jm , 

m j 

where the subscript p defines the hyperplane A p , which is to be located by 
adjusting the trial vector A, 



The adjustment of the tual vector can be made in the direction deter¬ 
mined by the maximum rate of change in the cuteiion <t> p This is deter¬ 
mined by the partial derivatives of <j> P with respect to the direction cosines 
X„„ We then have 


(48) 


d<j>p 



w } a im 


3 


= l 


mq j 


where l mq are the direction numbers of a vector L, as shown m the figure 
The numerical values of l mq are merely the sums of columns of the original 
factor matrix F with positive or negative signs as defined by w, The ad¬ 
justment of A, should then be made in the plane of A s and L, either toward 
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L or away from it, but we have, so far, no measure of how large the displace¬ 
ment of A„ should be in this plane 

In order to obtain a simple graphical method of determining the amount 
of displacement of A s , we determine the vector E in the plane of A, and L, 
where E is orthogonal to A, Then E is evidently a linear combination of the 
given trial vector A„ and the vector L. We then have 

(49) E — L — sA Q , 

where the paiameter s is to be determined. Since A„ and E are to be orthog¬ 
onal, we set 

(50) A q E = 0 
oi, by (49), 

(51) A„(L - sA v ) = 0 , 
and hence 

(52) A q L = sAl 
But since A a is a unit vector, we have simply 

(53) s = A„L , 

so that s is numerically determined as meiely the scalar product of A, and 
L By (49) and (53) the vector E is then located with direction numbers 
e mg Let C be defined as the vector E normalized with direction cosines 
c mq We then have A, and C as two orthogonal unit vectors 
The projections c,, are next computed from 

(54) Cjq — djmCmq r 

m 

and a plot is made of v,q against c mq From this plot the displacement of A, 
is determined When A q has been adjusted to a new position, it can again 
be regarded as a trial vector to be adjusted. It has been found that each 
primary reference vector can be located approximately in five or six adjust¬ 
ments of this land The fine adjustments should be made by a whole set of 
plots of pairs of columns of v 3P , which aie the elements in the oblique factor 
matrix V The present method will locate some, or perhaps most, of the 
primary reference vectors, while some of them may need final adjustment 
by the complete set of diagrams from the columns of V The present method 
seems to save a great deal of labor in solving the rotational problem. 
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Outhfoe of computations 

(1) When a correlation matnx has been factored into a factor matrix F, 
the rotational problem can be solved by the present method. A useful way 
to stait is to select a test vector Normalizing this test vector gives the unit 
trial vector A„ 

(2) Fmd the test projections v Jq on A„ and tabulate them 

(3) Place a strip adjacent to the column of projections v jq and mark on 
this strip negative signs for all tests which are to be reversed in sign ac¬ 
cording to the plan of Table 11 In the interpretation of this table one must 
make some decision as to what are to be regarded as large or small projec¬ 
tions In the first few trials these bands should ordinarily be rather wide, 
but in the last few trials the bands may be narrower so that a projection is 
regarded as small if it is numerically smallei than 20 

(4) Place this strip of signs adjacent to each column of F and determine 
the weighted sums (equation [48]), 

rCj a jm — l mq . 


(5) Find the scalar product (equation [53]), 


— A q L 


= 2 >„ 


(6) Find the diieetion numbers e mq of the vector E (equation [49]), 

li — L — sA q 

(7) Normalize E to find direction cosines c mq of the unit vector C 

(8) Check oithogonality by the relation A a C = 0 

(9) Find projections c, m of the test vectors on C 

(10) Plot v, q against Cjm 

(11) Locate a new trial vector as a linear combination of A a and C, as 
in the two-dimensional rotations 

(12) Find the projections on the new trial vector, and repeat the process 
if necessary 

In the interpretation of Table 11 it may be useful occasionally to assign 
positive or negative signs to the extremes and\to ignore some of the test 
vectors that are already in the hyperplane These test vectors that are m 
or near the hyperplane are then given weights, w, = Q When this is done, 
there are three weights for Wj, namely, +1, —1, or 0 

When one or more lesidual planes are to be located, it is probably best to 
choose one of the last few centroid axes as trial vector When a primary 
reference axis is to be located, it is convenient to choose a normalized test 
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vector as the trial vector The trial vector for the next plane should be a 
normalized test vector chosen fiom the tests that have near-zero projections 
on the reference axes that have already been determined In this manner 
one can avoid starting with a trial vector that moves into the position of 
one of the reference vectors already located 

Numerical example 

In order to illustrate the method we shall use actual data Table 12 is 
reproduced from “Psychometric Monograph,” No. 2, page 91. + This is a 
factor matrix for twenty-one tests and eight orthogonal factors, computed 
by the centroid method 


Table 12 
Factor Matrix F o 



I 

ir 

III 

iv 

V 

VI 

VII 

VIII 

If 

1 

51 • 

- 27 • 

- 26 

u 

— 

13 - 

- 09 - 

- 25 


10 

5102 

2 

62 • 

- 31 

12 

06 

— 

16 

12 - 

- 27 

— 

07 

6163 

3 

63 ■ 

- 09 

- 10 ■ 

- 09 

— 

09 

08 ■ 

- 21 


17 

5106 

4 

47 

28 

08 

- 14 

— 

33 ■ 

- 17 

17 

— 

10 

5020 

5 

38 

06 

22 • 

- 14 

— 

23 

10 ■ 

- 11 

— 

17 

3199 

6 

40 

19 

15 

- 19 

— 

34 

- 28 

14 

— 

07 

4732 

7 

66 

40 

11 

39 


22 

- 10 

- 09 


13 

8432 

8 

60 

43 

12 

27 


23 

- 19 • 

- 09 


06 

8490 

9 

70 

28 

26 

27 


28 

- 10 

- 10 

— 

08 

8137 

10 

63 

26 

- 29 

- 31 


19 

04 

- 17 

— 

10 

7213 

11 

59 

22 

- 26 

- 34 


23 

07 

- 03 

— 

09 

6465 

12 

53 

29 

- 19 

- 19 


20 

- 15 

- 09 


04 

5094 

13 

48 

- 44 

29 

- 18 


16 

16 

16 

— 

08 

6237 

14 

40 

- 50 

41 

- 25 


17 

18 

11 


08 

7204 

15 

46 

- 47 

42 

- 22 


12 

12 

08 


13 

7094 

16 

57 

- 26 

- 32 

19 

— 

05 

- 23 

17 

— 

12 

6297 

17 

58 

- 21 

- 42 

21 

— 

08 

- 24 

18 

— 

08 

7038 

18 

64 

- 21 

- 14 

18 


04 

03 

18 

— 

13 

5575 

19 

69 

13 

- 08 

15 

— 

15 

29 

19 


10 

6746 

20 

65 

23 

06 

21 

— 

13 

23 

08 


07 

0042 

21 

65 

.04 

- 16 

07 


10 

26 

09 


14 

5599 


Trial 1 is represented by one of the sections of Table IS We start with 
one of the test vectors, chosen arbitiarily This is test 14 in the present 
battery Its direction numbers are recorded in the first column in Table IS 
This vector is normalized by the multiplying constant, as shown at the bot¬ 
tom of the first column, and we then have the direction cosines of the first 
tual vector, A, as shown m the next column The projections of the test 
vectors on this tual vector are listed in column F, trial 1, of Table 14 Here 
it will be assumed that the structure is a positive manifold, and hence the 
primary reference axes aie assumed to be positive rather than bipolar Ac- 

* L L Thurstone and Thelma Gwinn Thui stone, Factorial Studies of Intelligence 
(Chicago University of Chicago Press, 1941) 
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cording to Table 11, we should reverse the signs of the test vectors with small 
positive projections in this column V We do this by reversing the signs of 
all test vectors whose positive projections in this column are less than 20 
The i eversals are indicated by the negative signs m the column adjacent to 
the piojections v 


Table IS 


Trial 1 



Test A 

14 A 

L 

e 


G 

I 

40 47 

— 

63 -2 

20 

- 68 

II 

- CO - 59 

-3 

07-1 

10 

- 34 

III 

41 48 

2 

46 

86 

27 

IV 

-25-29 

— 

36 

61 

19 

V 

17 20 


03 - 

64 

- 20 

VI 

18 21 

2 

35 1 

65 

51 

VII 

11 13 


10 - 

33 

- 10 

VIII 

08 09 


25 - 

05 

- 02 

Norm 

S 

1 178184 

3 3354 



310009 


Trial 2 


A e A L t c 


I 

35 

34 

- 89 

-1 22 - 

- 56 

II 

- 65 - 

65 

- 59 

04 

02 

III 

53 

52 

1 28 

77 

35 

IV 

- 26 - 

,26 

- 78 

- 53 - 

- 24 

V 

16 

16 

1 39 

1 23 

57 

VI 

30 

30 

- 59 

- 88 - 

- 40 

VII 

11 

11 

- 16 

- 27 - 

- 12 

VIII 

09 

09 

09 

00 

00 

Norm 

S 

985664 

9793 


460092 


Trial S 



A. 

A 

L 


e 

C 

I 

26 

26 

_ 

09 

- 41 - 

- 20 

II 

- 64 - 

- 63 

— 

21 

58 

28 

III 

57 

56 

i 

58 

88 

43 

IV 

- 30 - 

- 30 

-i 

16 

- 79 - 

- 39 

V 

25 

25 


71 

40 

20 

VI 

24 

24 

-i 

15 

-1 45 - 

- 71 

VII 

09 

09 


12 

01 

00 

VIII 

09 

09 

— 

05 

- 16 - 

- 08 

O 

B 

986095 

1 2495 



488462 


Trial 4 



Ae 

A 

L 

t 


c 

I 

22 

21 

- 25 

4 

96 

94 

II 

- 57 - 

56 

- 85 

— 

09 

- 02 

III 

65 

64 

- 24 

-1 

11 

- 21 

IV 

- 38 - 

37 

68 

1 

18 

22 

V 

29 

28 

27 

— 

11 

- 02 

VI 

10 

10 

33 


10 

04 

VII 

09 

09 

40 


28 

05 

VIII 

07 

07 

67 


57 

11 

Norm 

! 977157 



190052 

S 

1 3648 







Trial 5 



A. 

A 

L 


£ 

C 

I 

28 

28 

3 

81 

3 01 

85 

II 

- 56 - 

5G 

— 

51 

1 09 

31 

III 

63 

63 

1 

76 

- 03 

- 01 

IV 

- 35 - 

35 

— 

34 

66 

19 

V 

28 

28 

1 

39 

59 

17 

VI 

10 

10 

— 

37 

- 65 

- 18 

VII 

09 

09 

— 

64 

- 90 

- 25 

VIII 

08 

08 

“ 

33 

- 56 

- 16 

Norm 

1 992925 



281954 

S 

2 8484 
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These negative signs are now copied at the edge of a strip which is placed 
adjacent to each column of the factor matrix in Table 12, and the weighted 
sums are determined These weighted sums are recorded in column L of 
Table IS, trial 1 The scalar product s is next computed and recorded as 
shown This enables us to compute the direction numbers e of the vector B, 
as shown m the next column. The normalizing constant for this vector is 
shown at the bottom of column e, and then we have the direction cosines 
of the unit vector C, as shown in the last column for this trial It is useful to 
check the computations to make sure that the vectors C and A are orthog¬ 
onal by computing their scalar product, which should vanish 


Table 14 



Trial 1 

Trial 2 

Trial 3 

Trial 

Trial 5 




















' 


0 

V 



*r 

V 

± 



V 

db 

C 

V 

± 

c 

V 


± 

1 

_ 

30 

17 

_ 

— 

42 

12 

_ 

— 

30 

06 


66 

- 01 


41 


03 


2 

— 

15 

.47 


— 

43 

43 


— 

29 

36 


56 

30 


47 


34 


3 

— 

37 

26 


— 

43 

25 

— 

— 

25 

18 

— 

61 

13 

— 

49 


17 


4 

— 

40 

05 

— 

— 

34 

- 04 



17 

- 11 


39 

- 06 


41 

— 

03 


5 

— 

13 

23 


— 

pyl 

21 

—- 

— 

01 

17 

— 

26 

.16 

— 

31 


18 


6 

— 

42 

09 

— 

— 

EH 


— 


25 

- 02 


29 

03 

— 

33 


05 


7 

— 

57 


— 

— 

24 

- 07 


— 

02 

- 10 


68 

- 11 


82 

— 

05 


s 

— 

07 

05 

— 

— 

18 

b J ‘ 1 1 



10 

- 10 


66 

- 08 


86 

— 

02 


9 

— 

55 

23 


— 

15 

12 

— 


08 

10 

— 

63 

11 

— 

83 


16 


■M 

— 

65 

11 


— 

26 



— 

04 

- 04 


56 

- 05 


64 

— 

01 


11 

— 

62 

17 

— 

— 

23 

06 

— 

— 

03 

03 

— 

52 

02 

— 

56 


06 

_ 

12 

— 

65 


— 

— 

13 

- 07 



11 

- 08 


48 

- 07 


58 

— 

03 


13 

— 

10 

76 


— 

12 

73 


— 

10 

70 


36 

67 


21 


70 


14 


01 

85 


— 

02 

83 


— 

05 

83 


26 

80 


09 


82 


15 

— 

01 

83 


— 

06 

81 


— 

03 

79 


33 

77 


.16 


RO 


Bill 

— 

47 

17 

— 

— 

44 

08 

— 

— 

24 

01 

— 

64 

- 04 


45 


01 


17 

— 

52 

08 

— 

— 

14J 



— 

28 

- 08 


68 

- 14 


47 

— 

09 


18 

— 

38 

33 


— 

47 

26 

— 

—• 

32 

19 

— 

67 

12 

— 

49 


17 


19 

— 

35 

23 


— 

67 

16 

— 

— 

44 

07 

— 

73 

- 02 


51 


03 



— 

33 

18 

— 

— 

56 

11 

— 

— 

32 

03 

— 

66 

- 03 


57 


02 

— 

21 


34 

24 



61 

18 


* 

43 

09 


69 

00 


47 


05 

— 


The projections of the test vectors on the unit correction vector C are 
next computed and recorded in column C, trial 1, of Table 14 

The projections v are then plotted against the projections c, as shown m 
Figure 7 The new plane is located as shown in this graph by its trace 
thiough the origin The normal to the tiace defines the new extended trial 
vectoi, A„ This new trial vector is a linear combination of the first trial vec- 
toi, A, and the correction vector, C The vector equation for the new trial 
vector is shown in each graph The parameter 18 in this graph is deter¬ 
mined leadily when the plot is made on cross-section paper The fine grada¬ 
tions of the cross-section paper are omitted in these diagrams 

The direction numbers of the new trial vector are recorded in the first 
















Figure 8 









Figure 10 
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column of the next section of Table IS, trial 2. This vector is noimaimed 
by the normalizing constant at the bottom of this column, and we then 
have the new unit trial vector, A, whose direction cosines are listed m the 
next column The notation A„ represents the new trial vector, but it is longer 
than a unit vector The first step m each trial is to leduce the new extended 
tnal vectoi to unit length 

The calculations pioceed here as before to the deteimmation of the cor¬ 
rection vectoi C for trial 2 The projections of the test vectors on the trial 
vector A and on the collection vector C aie computed and recorded in 
Table 1 ! +, trial 2 The plot is shown m Figure 8 The small positive projec- 



Figvre 11 

tions on the new trial vector which are to be reversed in sign for the weighted 
summation are shown in black on the previous graph, Figure 7 These re¬ 
versals can be anticipated before the new trial vector is determined 

This procedure is repeated, as shown in Figures 9 and 10, until the fifth 
ti lal, which produces the graph in Figuie 11 Since this graph does not indi¬ 
cate any impiovement to be gamed by moving the tnal vector in the plane 
of A and the correction vector C, we accept the trial vector A for this trial 
as a close appioximation to the desired reference vector 

It can be seen m Figure 11, as well as from the last column of Table 14, 
that theie are thiec tests which are exceptionally high on this reference 
vector These aie the thiee space tests of the test battery m this experi¬ 
ment In oidei to locate another pumary lefeience vector, we should start 
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by normalizing one of the test vectors, which have small projections as 
3hown m the last column of Table 14 . 

The same procedure was used to locate the residual plane in this test 
battery This was done by starting with the last centroid column of Table 12 
and by revel sing the signs of all test vectors which had positive projections 
on the eighth centroid axis This leads to an improved position for the resid¬ 
ual reference vector One trial with this method was sufficient to locate the 
residual leference axis for these data 

When the primary reference axes, as well as one or more residual lefei- 
ence axes, have been determined by this simple method, the final step is to 
plot the whole set of diagrams, one diagram for each pair of reference vec¬ 
tors Since there are eight dimensions in this problem, the final set of dia- 
giams will have twenty-eight graphs Inspection of these graphs enables 
one to determine any further small adjustments that may be necessary to 
locate the best position for the simple structure The present method should 
be regarded as adequate for obtaining an approximate location of the refer¬ 
ence fiame of a simple structure; but it should always be followed by a com¬ 
plete set of plots of paiis of columns of the oblique factor matrix, to make 
sure that the best location of the frame has been attained. The piocedure 
here described is peihaps the simplest of the rotational methods so far de¬ 
vised, and it should serve to eliminate a very large pait of the labor which 
has hitherto been necessary in locating a simple structure in multidimen¬ 
sional factor problems 

In the application of this method or similar methods to the rotational 
problem, it will be found that some configurations are dominated by the 
large variance of the first centroid as compared with subsequent factors and 
that this circumstance causes slow convergence in these solutions In deal¬ 
ing with such configurations it may be found useful to extend the test vec¬ 
tors as in the method of extended vectors The configuration is then unal¬ 
tered except for the fact that all the test vectors are extended so as to have 
unit projection on the first centroid axis The present method can be ap¬ 
plied to such a configuration in the same manner as here described In the 
location of the new trial vector, one has additional freedom, in that a streak 
of points on the diagram which are not radial can be brought into a hyper¬ 
plane directly by expressing the next trial vector as a linear combination of 
thiee vectors, namely, the given trial vector, the new correction vector, and 
the unit reference vector along the first axis All the test vectors have unit 
projection on the fiist centroid axis, and the new trial vector can then be 
determined in the same manner as in the method of extended vectors This 
piocedure gives the solution m two trials with the present example 

Another variant of the method heie described is to weight each of the 
co-ordinates of the coriection vector inversely as the absolute sum of the 
entries m the corresponding columns of the given factor matrix F The 
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same result can be attained by writing a new factor matrix in which the ab¬ 
solute sums of the columns are equal. This involves a distortion of the con¬ 
figuration, but the simple structure is nevertheless invariant under this kind 
of distortion The disadvantage of this procedure on experimental data is 
that the axes on which the variance is very much extended become so dis¬ 
torted that the residual variance is more difficult to differentiate from the 
significant variance which is to be interpreted. 

A double-group method of rotation 

It happens not infrequently in a plot of two columns of the oblique factor 
matrix that the points reveal a concentration which could be fitted by a line 
that does not pass through the origin. In previous chapters there have been 
a number of such cases A few examples will be listed here to show the situa¬ 
tion m which the double-group method is applicable. Figure 8 of chapter x 
shows three such possibilities in the same diagram In that figure there is a 
roughly linear arrangement of points that could be fitted by a straight line 
through points 1 and 3. A second line could be drawn through points 2 and 
3, while a third line could be drawn through points 1 and 2 All but two of 
the points in the figure would be near one or more of these three lines In¬ 
stead of locating the line through points 1 and S, we might choose the cen¬ 
troids of the two groups, 1, IS, 18, and 3, 15, 20, which would, m this case, 
give practically the same solution 

In Figure 5 of chapter x we have another case The two groups, 1 , IS, 18, 
and 3,15, 20, might be chosen in this diagram to define the next adjustment 
in plane C In Figure 5 of this chaptei the plane C might be defined by a 
vector that is set orthogonal to the two groups, 2, 8, and 1, 7 

The principle involved in this adjustment is to express the new vector Ai 
as a linear combination of the given unit vector A 0 and the two centroid vec¬ 
tors Ci and C 2 of the two groups chosen graphically We start, then, with the 
following notation 

A 0 = given unit reference vector , 

Ai = new unit vector to be determined , 

Ci = centroid vector for group 1 , 

Ci = centroid vector for group 2 , 

Li = fciCi, where L\ = 1 , 

Li = fc 2 C 2 , where L\ = 1 

The vectors Li and L 2 are collinear with the corresponding centroid vec¬ 
tors The new vector Ai is expressed as the linear combination 


(55) 


Ai = Ao + p\L\ + piLi , 
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where pi and p 2 are scalars to be determined The vectoi Ai must satisfy the 
two requirements of orthogonality with Li and Li Then we have 

(56) A 1 L 1 = 0 
and 

(57) A iL 2 = 0 
Substituting the linear combination for Ai, we have 

(58) (A 0 -(- piL\ + piLi)L\ = 0 , 
from which we get 

(59) AoZ/i + piLf + piLiLi = 0 
L x is a unit vector, and hence 

(60) LI = 1 

Let AoZn = Si and let L\Li = l. Then 

(61) Si + Pi -f Pil = 0 

For the second condition of orthogonality, we have, similarly, 

(62) (Aq + p\Li + piLi)Li = 0 , 

from which we get 

(63) si + pxl + pi = 0 , 


where si — A 0 Li Solving for p 2 , we get 


(64) 

and, by analogy, 

(65) 


V 2 


s iZ — s 2 
1 - P 


V i = 


sit Si 


When the parameteis pi and arc thus known numeiically, the new vector 
Ai can be determined. It is normalized to become the new unit reference 
vector, A p When the double-gioup method is graphically indicated, it is 
quite effective 



CHAPTER XV111 
SECOND-ORDER FACTORS 

First-order and second-order factors 

Most of the work that has been done so far m the development of factorial 
theory has been concerned with the factors obtained from the test correla¬ 
tions with or without rotation of axes for the selection of a suitable reference 
frame Factors that are obtained from the test correlations will be called first- 
order factors, whether they are selected so as to be orthogonal or oblique, 
We shall now consider the factors that may be determined from the correla¬ 
tions of the first-oider factors Factors that are obtained from the correlations 
of the first-order factors will be called second-order factors. Factors of this 
type seem to be of fundamental significance in the interpretation of cor¬ 
related variables 

Analysis of second-order factors and their relations to those of first order 
can be presented in several different ways We shall describe these two 
factorial domains in terms of a literal notation, a physical example, a dia¬ 
grammatic representation, a geometrical example, and the matrix equations 
relating the two domains 

Consider, first, a reduced correlation matrix for the tests whose rank is, 
say, 5 The factoring of this correlation matrix determines five aibitrary 
orthogonal unit reference vectors which may be denoted I, II, III, IV and 
V This orthogonal reference frame is arbitrary in the sense that it is de¬ 
fined by the method of factoring which happens to be used This reference 
frame will be regarded as fixed, and all other vectors will be defined m terms 
of this fixed orthogonal frame, which is designated by the subscnpt m Let 
it be assumed that a complete simple stiucture can be found m the test con¬ 
figuration and let the corresponding primary vectors be denoted A, B, C, 
D, and E (These are oidinarily denoted T A , T B , etc ) We shall assume 
that these primary traits are correlated in the experimental population 
Then the primary vectors in the test configuration will be separated by 
oblique angles, whose cosines are the correlations between the primary 
traits in the particular group of subjects studied Let these correlations be 
listed in a new correlation matrix of order 5X5, showing the correlations 
between primary factors This correlation matrix defines the second-order 
domain just as the correlation matrix for the tests defines the first-order 
domain 

The simplest case is that m which the five primary factors are uncorre¬ 
lated, m which case their correlation matrix is a unit matrix so that an 
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analysis of a second-order domain is not immediately indicated Next would 
be the case in which the reduced correlation matrix for the primary factors 
A, B, C, D, and E is of unit rank There are two types of interpretation for 
such a situation The correlations between the primary factors in a particu¬ 
lar experimental population may be due to conditions of selection of the 
subjects, and in this case the conelations would be of no more theoretical 
importance than the conditions of selection of the subjects If, on the other 
hand, the five primary functions, A, B, 0, D, and E, actually do have some 
parameter in common, then one would expect their intercorrelations to be 
of unit rank for different experimental groups of subjects that are selected 
in different ways In other words, the mere fact that a set of variables or a 
set of factors is correlated does not imply any scientific obligation to find 
“the” factors that account for the correlations, because the factors, if found, 
might turn out to be as incidental in significance as the conditions by which 
the subjects happened to be selected. On the other hand, the fact that corre¬ 
lations between variables, or between factors, can be caused by scientifically 
trivial circumstances does not guarantee that all correlations between vari¬ 
ables are of trivial significance If the con elations between the five primary 
factors in the present example should turn out to be of unit rank, then this 
circumstance merits a closer look because such a simplification would not 
often happen by chance If the correlations between the primary factors 
should turn out to be of unit rank for several different experimental groups, 
then we should have an obligation to ascertain the cause which must tran¬ 
scend the selective conditions 

Let it be assumed that the five primary factors do have a parameter p 
in common. Then the five primaries could be expressed in the form 

A = f(p, a) , D = f(p, d ) , 

B = f(P, b) , E = f(p, e) , 

C = f(P, c) , 

where each primary function is defined m terms of a parameter such as a, 
b, c, d, or e, which is unique, and also in terms of another parameter, p, which 
it shares with the functions that define the other primaries If there should 
happen to be conspicuous correlation between the parameters a, b, c, d, 
and e in the particular group of subjects, then the unit rank of the second- 
order domain would be disturbed. If the correlations of the primaries show 
unit rank, then, in addition to the parameters a, b, c, d, and e, a second- 
order parameter or factor p can be postulated It should be noted that we 
now have six parameters, namely, a, b, c, d, e, and p, and, since the rank of 
the test correlations is 5, it follows that these six parameters are linearly de- 
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pendent In fact, the parameter p is now a linear combination of the other 
five parameters We can express these relations by a set of parameters such 
as A, B, C, D, E, and p, in which p is a linear combination of the five primary 
parameters The five primaries are parameters descriptive of the first-order 
domain, and the parameter or factor p is descriptive of the second-order 
domain, which is of unit rank here The second-order parameter is a linear 
combination of the five primaries that are defined by the original test cor- 
lelations If some degree of consistency can be found for these parameters 
for different groups of subjects, then all these parameters should represent 
some aspects of the underlying physical or mental functions 

Consider, next, a set of correlated primaries, A, B, C, D, and E, in which 
the parameter p appears in the first order, as in the following example: 

A = f(p, a) , D = f(p, d ) , 

B = f(p, b), E - f(p) 

C = f(v, c) , 

The rank of the reduced correlation matnx of the tests would now be 5. 
The five primanes listed above would be correlated and of unit rank The 
second-order factor p would be determined from the correlations of the 
primaries In this case the communahty for the primary factor E would be 
near unity, thus showing that its total variance is common to the second- 
order factor p Hence the primary factor E of the first order and the factor 
p of the second order would be identical The presence, or absence, of the 
primary E could be determined by including, or excluding, a few tests in 
the batteiy We see, therefore, that the appearance of a factor m the first 
order or in the second order may depend on the battery of measurements 
taken Hence a factor should not be considered as intrinsically different 
because it appears in the second order This circumstance can be deter¬ 
mined by the selection of the test battery. 

On the other hand, a parameter which always appears m the measure¬ 
ments in association with some othei function would not appear as in the 
primary E, and it would be discovered experimentally, nearly always in the 
second order Such a limitation could be introduced by the physical nature 
of the attribute which the factor represents, so that in such a case the second- 
order factor would represent something fundamentally different from that 
of the first order A single-factor study is not likely to reveal whether a 
second-ordei parameter is fundamentally different from the parameters of 
first older or whether the differentiation is caused merely by the selection of 
the test battery 
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In the following example we have another combination of primaries 


A = f(p, a) , I) = f(p, d) , 

B = f{p, b), E = /(e) 

0 = f(T, e) , 

In this example the reduced rank of the test correlations would again be 

5. The correlations of the primaries would show unit rank for A, B, C, 

and D The factor E would be orthogonal to the rest ol the system, so that 
its row and column would have side correlations of zero The correlations of 
primaries would not be of unit rank if we consider the whole table of order 
5X5, but it would be of unit rank if we consider only the 4X4 table for 
A, B, C, and D Relations of this land can be found by inspection of the 
correlations of the primaries, and they may be indicative of the underlying 
order of the domain that is being investigated 

The principles of a second-order domain have been discussed here in 
terms of the simple case in which that domain is of unit rank, so that there 
is only one general second-order factor It should be evident that the or¬ 
ganization of the second-order paiameters can be of any rank and complex¬ 
ity For example, the rank may be higher than 1, and the second-order fac¬ 
tors may extend to all the pumaries or to only some of them The possi¬ 
bility of third- and higher-ordei factors must bo recognized; but their ex¬ 
perimental identification is of increasing difficulty the higher the oidei, be¬ 
cause of the instability of such a supeistructure on piaetically feasible ex¬ 
perimental data The number of second-order factors that can be deter¬ 
mined fiom a given number of lineally independent primary factors follows 
the same restrictive relations that govern the number of primary factors 
that can be determined from a given number of tests Thus, foi example, 
it is not to be expected that three second-order factors would be deter¬ 
minate from only five primary factors, foi the same reason that three pri¬ 
mary factors cannot be determined from five tests Furtheimore, it is en- 
tnely possible m the same data for the first-oider domain to give clear inter¬ 
pretation of a set of primary factors and for the second-order domain to be 
indeterminate or ambiguous in the same data 

In order to avoid misunderstanding, perhaps it should be remarked that 
m factor analysis we are using the tcim parameter in its universal meaning in 
science A parametei is one of the measurements that are used for describing 
or defining an object or event In statistical theoiy the teim parameter is fre¬ 
quently used in a more lestncted sense as descriptive of the universe, as con¬ 
trasted with a statistic, which is the conespondmg measuiement on a sam¬ 
ple We aie not using the term in this lestrieted sense 
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The box example 

In order to lllustiate the nature of first- and second-oidoi factois we shall 
make use of populations of simple objects or geometrical figuies and their 
measurable properties instead of dealing with these factors merely as logical 
abstiactions We have used a population of rectangular boxes and their 
measurable attributes to illustrate the principles of correlated primary 
factors, and we can use them also for the pi esent discussion, 

A random collection of rectangular boxes was represented by the thiee 
measurements, length (x), width (y), and height (s) {chapter vii) A list of 
measurements was prepared which could be made on each box, such as the 
diagonal of the front face, the area of the top surface, the length of a vertical 
edge, and so on Each of these measurements represented a test score, and 
each box represented an individual member of the statistical population 
The correlations between the measurements were computed and analyzed 
factonally as if we did not know anything about their exact nature. 
Each measurement was tieated as a test score of unknown factorial com¬ 
position As has been shown previously, the analysis revealed three factors 
in the correlations for the particular set of measurements used. The con¬ 
figuration showed a complete simple structure, and a set of primal y vectors 
was determined by the configuration These three pumanes represented the 
three basic parameters in terms of which all the test measurements had been 
expressed. 

The three primary vectors weie separated by acute angles whose cosines 
lepresented the correlations between the three basic parameters that were 
used in setting up the box example These three correlations could be as¬ 
sembled into a small correlation matrix of order 3X3 The physical inter¬ 
pretation of the positive corielations was that large boxes tend to have all 
their dimensions larger than those of small boxes In other words, if one of 
the dimensions of a box shape is, say, 6 feet, the other dimensions of the box 
aie not likely to be of the order of, say, 2 or 3 inches The table of correla¬ 
tions of the thiee primary factors, X, Y, and Z, could be represented by a 
single common factor This factor would be a second-order factor It would, 
no doubt, be interpreted as a size factor in the box example If this second- 
order size factor were denoted s, we should have four parameters for de¬ 
scribing the box shapes, namely, the three dimensions x, y, and z and the 
size factor s These four parameters or factors would be linearly dependent 
because the rank of the correlation matrix of the tests was 3 

In the case of the box example, a size factor or parameter could be deter¬ 
mined m the first order if desired For this pui pose we could use the first 
centroid axis, the major principal axis, or the volume vector, all of which can 
be easily defined in the first-order system of test vectors The four param¬ 
eters so chosen would also be lineai ly dependent If we wanted to use 
only three linearly independent parameters, including a size factor, that 
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could be done in the first order by choosing, say, the two ratios x/y — n and 
x/z—r^ as well as the volume vectoi v These three factors would be linearly 
independent, but they would be coi related. The latitude with which we can 
choose simplifying parameters for the box example is determined in part by 
the fact that three factors can nearly always be represented by a common 
factor, whereas this is not the case when the rank is higher than 3. 

Diagrammatic representation 

The relations between the first-order and the second-order domains can 
be represented diagrammatically as shown m Figures 1 , 2, and S In Figure 1 
we have a set of eight tests, whose correlations aie accounted for by five 



Orthogonal 7ests 

factors 


Figure 1 

primary factors, A, \B, C, D, and E, which are uncorrelated The factor A, 
foi example, is present in the common-factor variances of tests 1, 2, and 4 
The primary factor; E is present in the common-factor variances of all the 
tests, and hence E (would be called a general factor for the particular bat¬ 
tery Since it is orthogonal to all the other primary factors, it would be 
called an orthogonal general factor of the first order. In order to determine the 
nature of the factor E it would be necessary to study it in different test bat¬ 
teries so that one could predict with certainty when the factor would be 
present and when it would be absent from a test Since the primary factors 
are here represented as uncorrelated, the matrix of correlations of the pri¬ 
mary factors would be an identity matrix, and there would be no immediate 
provocation to investigate a second-order domain. 
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In Figure 2 we have represented a set of tests and five primary factors, 
A, B, C, D, and E (We are not here concerned as to whether the particular 
number of tests represented in this diagram is adequate for the determina¬ 
tion of five primary factors. The purpose of these diagrams is merely to 
show the nature of the relations between the two domains.) The rank of the 
correlation matrix of the tests would here be 5, which corresponds to the 
number of linearly independent primary factors In the present case we 
should find that the primary factors are themselves correlated The matrix 
of correlations of these primaries would be of order 5X5, and it would be of 
unit rank The correlations between the primary factors could therefore be 



Second order Ccrre/afed Tests 

factors factors 

Figure 2 

accounted for by a single general second-order factor that is denoted G If 
both the first-order and the second-order factors were to be used for the de¬ 
scription of the tests and the relations, we should have six parameters which 
would be linearly dependent because the rank of the correlations of the tests 
is only 5 In fact, the satuiation of each test with the second-order factor G 
would be a linear combination of the saturations of the test with the five 
primaries of the first order. None of the primary factors are general factors 
in this figure 

In Figure 3 we have a more complex relation, in that the correlation ma¬ 
trix for the primary factors would be of rank 2. One of the second-order fac¬ 
tors is here shown to be common to all but one of the primary factors; one 
of the second-order factors is a factorial doublet, m that it represents addi- 
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tional correlation between the primaries B and D, and the primary factor A 
is orthogonal to the rest of the primaries, so that it does not participate in 
the second-order domain This diagram is drawn merely to illustrate the 
variations in complexity that may be found m factorial studies 

The two types of general factors here shown in Figures 1 and 2 have some 
interesting differences. The general factor E of Figure 1 is independent of 
the other primary factors, while the general factoi G in Figure 2 is present 
in all the other factors Hence we must conclude that a second-order gen- 

© 



Second onfor 
foct-O’-S 


First c der 
/actors 


Figure 3 

eral factor is a part of, and must pai ticipate m, the definition of the other 
factors, while the oithogonal geneial factor E of Figure 1 is, by definition, 
independent of the other primary factors It is evident, theiefore, that a 
geneial second-order factor is likely to he of more fundamental significance 
for the domain in question than a general orthogonal first-order factor An 
orthogonal general factor of the first order might operate m a test without 
any gioup factor, wheieas a second-order general factor would operate, 
ordinarily, through the mechanism of some function that could be identified 
as a group factor, a primary factor, or a special ability 

The factor patterns coiresponding to the relations shown diagrammati- 
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cally m these figures are given in Tables 1, 2, and 3 Table 1 shows the factor 
pattern for Figure 1 Here the orthogonal general factor E is identified by 
the fact that all entries of its column are filled Table 2 shows the factor pat¬ 
tern for Figure 2 Here it is seen by the factor pattern that a group such as 
tests 1, 4, 5, and 7 have no primary factor m common and that hence then 
correlations would be determined only by the second-order general factor G 
The determinant of the correlations for these four tests (the tetrad differ¬ 
ence) would therefore vanish The second-order factor matrix is also shown 
in this table with only one factor G to correspond to this example 


Table 1 


Orthogonal Factors 



A 

B 

0 

D 

E 

1 

X 




X 

2 

X 

X 



X 

3 



X 


X 

4 

X 

X 



X 

5 




X 

X 

6 



X 

X 

X 

7 



X 


X 

8 




X 

X 


Table 3 


Second-Order Do¬ 
main of Figure 3 



P Q 

A 


B 

X X 

C 

X 

D 

X X 

E 

X 

F 

X 

G 

X 


Table 2 


Correlated Factors 



ABODE 

1 

X 

2 

XXX 

3 

X 

4 

X 

5 

X X 

6 

XXX 

7 

X 

8 

X X 

9 

X 



G 

A 

X 

B 

X 

C 

X 

D 

X 

E 

X 


The question might be raised as to whether both types of general factors 
could be present in the same battery. That seems possible In that case a 
simple structure could define the primary factors A, B, C, and D but not 
E in the particular battery of Figure 1. This factor could be assumed arbi¬ 
trarily to be orthogonal to the other factors, but then the line GE of Figure 2 
would be erased to correspond to the fact that E is orthogonal to the other 
factors One or more second-order general factors could be found in the cor¬ 
related primaries If the correlations of A, B, C, and D were of unit rank, 
another alternative would be to set E in such a relation to the other primary 
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vectors as to maintain the unit rank with the second-order general factor 
It might then be found that the vector E has non-vanishing projections on 
all the test vectors, m which case both types of general factor would be as¬ 
sumed to be a possible set of explanatory parameters for the battery m 
question It must be remembered that these various locations of the refer¬ 
ence frame for the explanatory parameters in both the first-order and the 
second-order domains have validity only in so far as they are suggestive of 
fruitful scientific interpretation If this is not the purpose, then the factorial 
resolution might as well remain in the arbitrary orthogonal factors produced 
by factoring the given test correlations—or, better still, by not doing the 
factoring at all, 

It might be asked how the correlations of a test battery can be resolved 
into a second-order domain of unit rank which is lower than the rank of the 
test correlations The transitions can be regarded geometrically The unit 
test vectors usually define a space of as many dimensions as there are tests 
When the reduced correlation matrix is considered, its rank is frequently 
lower than its order. Hence the reduction from the number of tests n to the 
number of primary factors r represents a reduction from the total variance 
of the tests to the common-factor variance The complete eon elation ma¬ 
trix for the primary factors represents a set of r unit vectors m as many di¬ 
mensions, the dimensionality of the common-factor space The reduced 
form of this matrix for the example of Figure 2 would have unit rank be¬ 
cause the side correlations are determined only by that which the primaries 
have in common, namely, the second-order general factor. 

Group factors and primary factors 

In Figure 4 we have a diagrammatic representation of a different kind of 
resolution of factors in the second-order domain and their relation to the 
primary factors. In this example the rank of the correlation matrix of tests is 
assumed to be five, as represented by as many primary factors, A, B, C, D, 
and E Let it be assumed that the correlation matrix for these five primaries 
is of unit rank The general second-order factor G then accounts for the ob¬ 
served correlations of the primary factors If the five linearly independent 
primary unit vectors and the second-order unit vector G are to be represent¬ 
ed in the same space, the dimensionality of this space must be six It is 
possible to locate in this augmented space anothei set of unit vectors, a, b, 
c, d , and e, which are mutually orthogonal and which are also orthogonal to 
the unit vector G Then we have the orthogonal reference frame G, a, b, 
c, d, and e, which defines the six dimensions of the first- and second-order 
factors but not the test space. The five linearly independent primary factors 
define a five-dimensional space corresponding to the rank of the test corre¬ 
lations, and this space is a part of the total six-dimensional space of this 
representation 



SECOND-ORDER FACTORS 


421 


The unit vector a is a linear combination of the unit vector G and the 
primary vector A The relation is similar for the other primary vectors The 
primary vectors A, B,C,D, and E are correlated and of unit rank, whereas the 
vectors a, b, c, d, and e are arbitrarily set orthogonal to each other In gen¬ 
eral, if the rank of the test correlations is r and if the correlations of the 
primaries are of unit rank, then the primaries define a unit vector G for a 
general second-order factor in an augmented space of dimensionality (r+1) 
and also a set of r mutually orthogonal unit vectors, each of which is in the 
plane of the second-order general factor and one of the primaries These 
vectors are arbitrarily set orthogonal to the general second-order factor, and 



Second order Primary factors Group feefors 

genera/ factor 

Figure 4 

they are called group factors In Figure 4 the primary factors are denoted 
A, B, C, D, and E, and the group factors are denoted a, b, c, d, and e With 
this resolution we have (r+1) linearly dependent factors, which represent 
the test correlations of rank r This type of resolution is preferred by some 
students, who use the reference frame G, a, b, c, d, and e because it is orthog¬ 
onal rather than the frame G, A, B, C, D, and E, winch is oblique. 

A general second-order factor 

The algebraic and computational relations between the first-order and 
the second-order domains will be shown for the case of a single general 
second-order factor because of the interest of this case for the psychological 
controversies of the past forty years about Spearman’s general intellective 
factor. The algebraic and computational relations to be shown can be gen- 
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eralized to second-order domains of higher than unit rank It must be re¬ 
membered, however, that the restnetion of our discussion to unit rank foi 
the second-order domain does not m any way imply that such low rank is 
always to be expected The methods of analysis can he readily extended to 
a second order of higher rank when the data indicate a determinate second- 
order configuration. In any case the second-order rank should be consider¬ 
ably lower than the rank r of the first-order factors m order to justify inter¬ 
pretation. 

The primal y vectors constitute a set of r lmeaily independent unit vec¬ 
tors that define a space of dimensionality equal to the rank of the test corre¬ 
lations In order to repiesent a general second-order factor as a unit vector 
in the same configuration it is necessary to augment the dimensionality to 
(r + 1) dimensions A second-order domain of rank 2 would thus require an 
augmented space of dimensionality (r+2) The projections of the test vec¬ 
tors on these additional vectors in the augmented space can, however, be ex¬ 
pressed as linear combinations of the test projections on the primary vec¬ 
tors or on any set of r lmeaily independent vectors in the common-factor 
space The procedures for determining these saturations will be shown with¬ 
out wilting explicitly the (r+1) co-ordmates of the second-order unit vec¬ 
tors in the augmented space 

The present discussion is confined to factoiial data that satisfy two con¬ 
ditions, namely, (1) that a complete simple stiuctuie is revealed in the test 
configuration and (2) that the second-order correlation matrix is of unit 
rank These methods can be adapted to the analysis of less than r primary 
factors, and the methods can be adapted to higher second-order rank 

One of two objectives will be assumed, namely, (1) to determine the pro¬ 
jections (saturations) of the tests on the second-order factor in addition to 
the piojcctions on the primary refeience vectors or (2) to determine the pro¬ 
jections on the second-oider factor and also on the orthogonal gioup factors 
It will be convenient to discuss the algebraic relations under four cases be¬ 
cause of the different computational routes that may be chosen These four 
cases are as follows 

Case 1 Transformation from F to Y, including the column vector G 

This transfoi mation is shown in rectangular notation in Table 4 for the 
equation 

(T) F + = V 

in which the matrix V, p has an extra column for the second-oi der factor Q 
with elements v ]a , which may also be denoted r,„ because these aie the corre¬ 
lations between the tests j and the general factoi G The transformation 
matrix + mp is identical with A mp except for the added column G with ele¬ 
ments ipma, which are to be determined Consider the matux T as an extern 
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sioii of the factor matrix F. The rows of T give the direction cosines of the 
primary vectors Tt with elements tt m The same transformation gives 

(2) T t JF mp = Tip , 

which is the diagonal matrix D except foi the first column Applying the 
transformation we have 

(3) 1 tni'Fino = Tto i 


Table 4 

Case 1 Ti ansfovmahon ftom F to V, Including the Column Vector G 



Case 2 T> ansformahon from F to U, Including the Group 
Factors and the Column Vector G 
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where r ts is the first column of V < and its elements are the correlations of the 
primary factors with the general factor O These are known from the factor¬ 
ing of the unit-rank correlation matrix for the primaries. Then 

(4) 'k rng ~ T iff lg t 

and, since T = DA -1 , we have 

(5) 'P.m = A D~ l r la , 

from which the first column of \P m( , can be computed Hence the column 0 
of the augmented oblique factor matrix V becomes known 

Case 2. Transformation from F to U, including group factors and general 
factor G 

Here the computation starts again with the orthogonal factor matrix F, 
and the objective is to determine the saturations of the tests j with the r 
group factors and the second-order general factor O This transformation is 
also shown m Table 4 in rectangular notation by the equation 

(6) F } JSr mill Uju> j 

where U is the factor matux, showing the projections of tests j on group fac¬ 
tors and general factor 0 These (rfi-1) mutually oithogonal factors will be 
designated by the subscript w The first column of this matrix is again the 
column of correlations r„. If the same transformation is applied to the ma¬ 
trix T for the pi unary vectors, we have 

(7) r,A, = u„„, 

which is also a diagonal matrix except for the first column, which contains 
the correlations rt„ between the primary factors and the second-oider gen¬ 
eral factor C These saturations can be determined from the unit-rank cor¬ 
relation matrix TT' = R, for the primaiy factons * Consider the first row 
of U t The two entries m this row show the direction cosines of T A m terms 
of the orthogonal fiame G, a, b, and c The primary vector T A is a lmeai 
combination of the two oithogonal unit vcctois G and a Hence, when r u is 
known, we have 

(8) r\ 0 + 1 ^ 2=1 

* Elsewhere we have denoted this matrix R pq , but we are here using the subscript t for 
the primary vectors Ti and reset ving the subscripts p and g for the primary refeience 
vectors A, B, and C Hence the correlations of the primary factors are here denoted 
fit instead of R v „ 
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or 

(9) r\„ + u\ a = 1 , 

so that the element u Aa is known The other diagonal elements of U , are 
determined in the same way, so that, for example, 

(10) r%„ + u% b = 1 . 

When the matrix U t is known, we have, by (7), 

(11) Tv = T~ l Ut 

or 

(12) ^ m\u — A D-'Ut, 

so that the transformation 'I' m u> is known The saturations of tests j on the 
second-order general factor G and the group factors w can then be computed 
The transformation matrix Tv represents a rigid rotation fiom one or¬ 
thogonal frame to another orthogonal frame, and hence this transformation 
matrix must be orthogonal by rows A fourth row could be added to Tv 
for a fourth orthogonal unit vector IV with cell entries which normalize 
each column Then we should have an orthogonal matrix of order 4X4 

Case S Transformation from, V to U, including the group factors and column 
vector G 

Here it is assumed that the computations are to be made from the oblique 
factor matrix V In Table 5 we have the transformation equation in rec¬ 
tangular notation, namely, 

(13) V, p ¥ pw = U)v >, 

which gives the satuiations of the tests j on the group factors and on the 
general factor If the factor matrix V is extended to include the primary 
vectors T t , we have the diagonal matrix D Applying the same transforma¬ 
tion to D, we have 

(14) D (J ,Tv> = U h 
so that 


(15) 


TV = D-'Ut 
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When the elements of U t have been deteimined as for Case 2, the tiansfor- 
mation can be wiitten by merely adjusting the rows of U , by the multi¬ 
pliers of Djp The ti ansformation is then known 


Table 5 

Case 8 Transformation from V to U, Including 
the Ghoup Factors and the Column Vector G 




U 

G _ a _£ c 


£ 

X 

X 



5 

£ 





* 



X 


H 


Case 4 Transfot mation /> om V to Column Vcclot G 


A B C 



D 



Case 4 Transfoi mation from Y to column vectoi G 

This is the simplest case and perhaps the most useful as legards the 
second-order domain The matrix V is known in determining the simple 
structure and the primaries The saturations of the tests j on the second- 
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01 dei general factoi G are of intei est, and these can be determined as linear 
combinations of the columns of V Here we have the transformation shown 
in Table 5, namely, 

(16) = T>t 

Applying the same transformation to D lp , we have 

(17) D lp <Sr po = r la 

The elements of the column vector r t „ are known from the correlation ma¬ 
trix TT' — Rt of the primaiies Then 

(18) = D^r t „, 

and hence the column vectoi ^ vg is known In computing, it is necessary 
only to multiply the elements r la by the con esponding diagonal elements of 
to determine The desired column vector, r 10 , can then be deter¬ 
mined. 

A trapezoid population 

In previous studies of factorial theory it has been found useful to illus¬ 
trate the pnnciples by means of a population of simple physical objects or 
geometncal figuies The box population was used to illustrate three corre¬ 
lated factors and their physical interpretation In the present case we want 
four factors in the first-order domain, which, by their correlations of unit 
rank, determine a general second-order factor The correlations of three 
variables can nearly always be accounted for by a single factor, and hence it 
seems bettei to choose a four-dimensional system in which the existence of a 
second-older geneial factor is more clearly indicated by the unit rank of the 
correlations of four primary factors For the present physical illustration 
we have chosen a population of trapezoids whose shapes are determined by 
four primary parameters or factors 

The measurements on the trapezoids are indicated m Figure 5 The base 
line is bisected, and the length of each half is denoted by the parameter c. 
An ordinate is erected at this mid-pomt, and its length is h. This ordinate 
divides the top section into two parts, which are denoted a and b as shown. 
These foui parameters, a, b, c, and h, completely determine the figure The 
test battery was represented by sixteen measurements, which are drawn m 
the figure The parameters a, b, c, and h are given code numbers 1, 2, 8, 
and 4, respectively Variables (12) and (18) are the two areas as shown The 
sum of (12) and (18) equals the total area of the trapezoid In general, each 
of these measurements is a function of two or three of the parameters but 
not of all four of them, and hence we should expect a simple structure m this 
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set of measurements There is a rather general impression that a simple 
structure is necessarily confined to the positive manifold In order to offset 
this impression we included here three additional measuies, which extend 
the simple structure beyond the positive manifold These three additional 
measures are as follows 

14 = (1)/(2) = a/b , 15 = (2)/(3) = b/c, 16 = (l)/(3) - a/a. 

These three measures will necessarily introduce negative saturations on 
some of the basic factors 

In Table 6 we have a list of dimensions for a set of thu ty-two trapezoids. 
These will constitute the trapezoid population Each figure was drawn to 



Figure 5 


scale on cross-section paper, and then the sixteen measurements were made 
on each figure These constituted the test scores for the present example 
In setting up the dimensions of Table 6 the numbers were not distributed 
entirely at random To do so would tend to make the correlations between 
the four basic parameters, a, 6, c, and h, approach zero, and this would lead 
to an orthogonal simple structure in which there would be no provocation 
to investigate a second-order domain The manner in which the generating 
conditions of the objects determine the factorial results will be discussed in 
a later section Table 6 was so constructed that, in addition to the four basic 
parameters, there was also a size factor, which functioned as a second-order 
parameter m determining correlation between the four pnmai y factors in 
generating the figures. 

The product-moment corielations between the sixteen measurements for 
the thirty-two objects were computed, and these are listed in Table 7 This 
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correlation matnx was factored by the gioup centroid method, and the re¬ 
sulting factor matrix F is shown in Table 8 The fourth-factor residuals aie 
listed in Table 9, which indicates that the residuals are vanishingly small 
Applying the rotational methods to the configuration, we found the tians- 
formation matrix A of Table 10, which produced the oblique factor matnx V 
of Table 11 In this matrix wc are now concerned with all but the last col- 

Table 6 


Trapezoid Parameters 



a 

b 

C 

h 

1 

i 

2 

1 

2 

2 

i 

2 

1 

4 

3 

i 

2 

3 

2 

4 

i 

2 

3 

4 

5 

i 

3 

1 

2 

6 

i 

3 

1 

4 

7 

i 

3 

3 

2 

8 

i 

3 

3 

4 

9 

2 

2 

1 

2 

10 

2 

2 

1 

4 

11 

2 

2 

3 

2 

12 

2 

2 

3 

4 

13 

2 

3 

1 

2 

14 

2 

3 

1 

4 

15 

2 

3 

3 

2 

16 

2 

3 

3 

4 

17 

2 

3 

3 

3 

18 

2 

3 
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5 
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3 

5 
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2 

4 

3 

3 
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2 

4 

3 
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23 
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5 

3 

24 

2 

4 

5 

5 

25 

3 

3 

3 

3 

26 

3 

3 

3 

5 

27 

3 

3 

5 

3 

28 

3 

3 

5 

5 

29 

3 

4 

3 

3 

30 

3 

4 

3 

5 

31 

3 

4 

5 

3 

32 

3 

4 

5 

5 


li mn When pan s of columns of the factor matrix V are plotted, we have the 
configuration shown in the diagrams of Figure 6, in which a simple structure 
is clearly indicated The cosine of the angle between the reference vectors is 
indicated on each diagram of Figure 6. These cosines were obtained from 
the relation C = A'A, as shown in Table 12 

So far m the analysis we have found that foui primary factors account for 
the correlations, and this corresponds to the fact that we used four param¬ 
eters m setting up the trapezoid figures. The four primary factors are cor¬ 
related, as indicated by the obliqueness of the reference axes m the dia- 





0 

° H 

O 

0 0 

0 

0 

c • a 


cos*- /5 


H 


Frannii 6 


431 


3 





cos — // 









inn 




4 31 


432 


MULTIPLE-FACTOR ANALYSIS 


grams of Figure 6 The next step is to determine the correlations between 
the primary factors that correspond to the pnmary reference axes For this 
puipose the mveise of the matrix C is computed, as shown in Table 12 From 
the diagonal values of this matrix are found the numerical values of the 
diagonal matnx D, which is also shown m Table 12 The inverse of this diag¬ 
onal matrix is also listed These numerical values are merely the reciprocals 
of the entries m D 
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In Table 13 we have the conelation matrix R,, showing the correlations 
between the pumary factois These are the cosines of the angles between 
the primary vectors It can be seen by inspection that this matrix is close 
to unit rank, which indicates that a single general second-order factor can 
be postulated to account for the correlations between the primary factors 
The saturation of each pi imary factoi with this second-older general factor 
was determined by one of the special formulae for unit rank, and the satura- 
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tions are listed in the column vector r is The interpretation is, for example, 
that the primary factor A has a correlation of 71 with the second-order 
general factor O. The closeness of the correlation matrix to unit rank is 
shown by the small side correlations in the residual matrix of Table IS The 
diagonal values of the residual matnx show that pait of the total variance 
of each primary factor which it does not share with the general second-order 
factor If the diagonals of this matrix vanished completely, then the pri¬ 
maries would have their total variance in common, and the original reduced 
correlation matrix for the tests would have been of unit rank. 

The saturation of each test with the second-ordei general factor was de- 
tei mined as a linear combination of the columns of the oblique factoi matnx 
V of Table 11 The transformation of equation (18) was used, and the nu¬ 
merical values of were listed m Table IS Column G of Table 11 was then 
computed by equation (16) 

The second-order general factor G can be mterpieted in this example as a 
size factor, and it also indicates that, in generating the thirty-two figuies, 
the four parameters a, b, c, and h were not allowed to take entirely independ¬ 
ent values In other words, either the extreme foi ms of figures did not occui, 
or else they were used only occasionally If the four parameteis had been 
allowed to take entnely independent values, then there would have been an 
appreciable number of figures m which one of these paiamoteis had an un¬ 
usually small value, while some other parameter had some unusually large 
value. This interpretation of the second-order general factor leads to a con¬ 
sideration of what we shall call generating parameters The present geo¬ 
metrical example illustiates the type of factorial organization that is lepre- 
sented chagrammatically in Figure 2 The problem of inteipreting the four 
primary factors can be solved m this case without investigating the second- 
order domain But if the conelations between the primary factors show un¬ 
expectedly low rank, then this fact can be utilized factorially in gaining fur¬ 
ther insight into the conditions under which the objects weie generated 
The four primary factors here identified by the simple structure were the 
four parameters that weie used in setting up the problem 

Generating parameters 

In addition to the principle of simple structure foi the descnption of each 
individual object, we may consider an extension of this principle to the 
problem of describing the manner in which the measured objects were gen¬ 
erated Other things being equal, we should pieferaset of descriptive paiam¬ 
eters that give some indication of the conditions that were operative in 
producing the objects. To the extent that a factoi analysis can throw some 
light on the conditions that were responsible for pioducmg the objects and 
their measurable chaiactenstics m addition to the description of each indi¬ 
vidual object, both by some simplifying set of paiameteis repiesentmg cans- 
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ative factois, the factorial methods become even more useful as tools in sci¬ 
entific work 

The numencal values of the trapezoid parameter in Table 6 defined 
thirty-two figures of various shapes The method of constructing the table 
of four measures for each figure determined whethei one ormoie second- 
order factors would be present and also whether each of the pumaries would 
be equally or diffei cntly represented m the second-order factor The fac¬ 
torial result could be altered indefinitely by the mannei in which the objects 
were generated m constructing Table 6. Since it is the object of factor anal¬ 
ysis to reveal the underlying order in the domain, it is an essential part of 
the numerical example to show that there is a relation between the generat¬ 
ing principles and the factorial results 

The first column of the table contains the tluee hneai measurements 1, 
2, and 3 Suppose that these were inserted in the column entirely at landom 
Assume that each column was similarly constructed by distnbuting a set 
of measurements entirely at random Then we should expect zero conela- 
tions between the four primaries, Ta, Tb, Tc, and Th The conelation ma¬ 
trix for the foui primaly factois would be an identity matrix, and it would 
not be factored because the primary factors would be statistically independ¬ 
ent Theie would be no second-oidcr factor present 

If, for each one of these thirty-two figuies with uncorrelated primaries, 
we should draw another one similai in proportions but with twice the area 
and another one with similar proportions but three times the area, then we 
should have a set of ninety-six figures, consisting of three sets that have 
sinnlai shapes but diffei ent sizes If this new set of ninety-six figures were 
analyzed factonally with the same battery of sixteen measurements, we 
should find the same primary factors, but they would be coirelated Fur- 
theimoie, the coirelations of the primary factors would all be the same, so 
that we should have a correlation matrix for the primary factors with uni¬ 
form side correlations The reduced conelation matrix would have unit 
rank, and all the four pnmanes would have the same saturation on the sec- 
ond-ordei general factor This would be a situation with a second-order 
general factor which has a uniform effect on all the primaries Here, again, 
the factorial lesult would be determined by the manner m which the objects 
were generated 

Suppose that a group of peisons was asked to diaw some tiapezoids of 
arbitral y shapes and that these trapezoids were assembled as a population 
of figuies to be measured and analyzed factonally Then we should almost 
certainly intioduce a second-order size factoi because our subjects would 
piobably unwittingly draw the figures so that the several dimensions of each 
figure would be at least roughly of the same general order of magnitude 
Some of the subjects might draw tiapezoids of the general size, of say, 5 or 
6 inches, while other subjects might draw figures only 1 or 2 inches across 
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Very few would produce trapezoids that are 1 or 2 inches wide and 10 inches 
tall. In other words, since some subjects would draw big figures and others 
small ones and since they would probably produce very few extreme figures, 
there would be strong correlation between the primary factors, and these, 
in turn, could be analyzed factorially into secondary factors In this situa¬ 
tion the rank of the correlations of the primary factors would probably not 
be exactly 1, but the inference could certainly be drawn from the factorial 
result that secondary factors were operative to produce some big figures and 
some small ones in addition to the primary parameters that define the in¬ 
dividual figures 

The interpretation of the second-order factor as a size factor in the trape¬ 
zoid example should be distinguished from the size factor that could be 
chosen as a parameter in the first-order domain. If one of the measurements 
had been the total area of the trapezoid, it would have been represented by 
a test vector in the middle of the configuration, since it would be affected by 
all four of the generating parameters that were used and which appeared in 
the simple structure The total-area test vector could be normalized to a 
unit vector, and it could be used as one of the parameters for describing the 
tiapezoids It would not be identical with the second-order size factor, but 
they would be closely related. Whether a size factor appears as a first-order 
or a second-order factor depends on the restrictive conditions under which 
the figures or objects are produced or selected and also on the selection of 
measurements for the test battery It is interesting to note that here the 
lesults would indicate either that the thirty-two trapezoids had been sys¬ 
tematically selected by some restrictive conditions or else that the objects 
themselves had been generated under some lestiictive conditions 

When the factorial results are clear in both the first-order and the second- 
order domains, inferences can sometimes be drawn concerning the generat¬ 
ing conditions that produced the individual parts of the objects Such in¬ 
ferences can be the basis for formulating hypotheses that can be investigated 
further either by factorial methods or by moie directly controlled experi¬ 
ments 

Incidental parameters 

So far we have considered the primary factors determined by a simple 
stiucture as representing parameteis that can be given some scientific inter¬ 
pretation in terms of concepts that are fundamental for the domain in ques¬ 
tion In using the simple-stiucture solution, which leads sometimes to the 
second-order domain, we have tried to avoid using arbitrary parameters 
whose only merit is that they sei ve in the condensation of numerical data 
We have tried to find in the primary factors a set of paiameters which not 
only describe the individual measurements but also reveal something about 
the undeilymg order in the domain In looking for meaningfiil parameters 
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of this kind it would be an error to assume that all the factors have signifi¬ 
cance that transcends the particular experiment or the particular group of 
subjects It would be strange indeed if factor analysis were immune from 
the distracting circumstances of the particular occasion The experimenter 
must try to distinguish that which is invariant and which transcends the par¬ 
ticular expenmental arrangement or the particular experimental group of 
subjects from that which is local and incidental to the particular occasion 
In factor analysis we are not relieved of this difficult task any more than 
in other foims of scientific experimentation. In order to focus attention on 
this circumstance, it might be well to distinguish the primary factors which 
represent the invariants for which we are really looking from those primary 
factors which, though genuine as regards the explanation of the test vari¬ 
ances, are local and of significance only for the experimental group 01 the 
particular occasion Primary factors which characterize only a particular ex¬ 
perimental group 01 a particular situation may be called incidental factors to 
distinguish them from the invariants which are normally the object of scientific 
experimentation Incidental factors may appear in the first-order or in the 
second-order domain 

A few examples will serve to illustrate the manner m which incidental 
factors may appear as primaries in factorial analysis In addition to the 
primary factors that would be found m different groups of subjects, we 
might find primary factors that are unique for the particular occasion Sup¬ 
pose that an exceptionally good examiner who is skilled m obtaining good 
rapport with the subjects should give a part of the test battery to a part of 
the experimental group A primary factoi might appear for this group of 
tests, and the investigator might be at a loss to explain it because he would 
be thinking about the nature of the tests and he would try to find something 
common m the psychological nature of these tests It might not occur to 
him that this is the very group of tests that was administered by the experi¬ 
enced examiner Such a factoi would probably be left without mterpieta- 
tion in the final results, or the interpretation might be one that would not 
be sustained m a subsequent expeument with different subjects and differ¬ 
ent examiners Incidental factors are almost certainly present in every 
study Hence the investigator should feel free to leave without interpreta¬ 
tion those primary factors which do not lend themselves to rather clear sci¬ 
entific interpretation Even then the interpretation should, at first, be m 
the nature of a hypothesis to be sustained, if possible, by subsequent fac¬ 
torial studies. The fact that all the variances are not adequately accounted 
for m the interpretation has led some students to conclude that the whole 
result should be discarded, but that is not the case It is quite possible to 
make an important discoveiy concerning the primary factors that are opera¬ 
tive in an experiment, even though the major part of the common-factor 
variances remains unexplained It is assumed, of course, that such a finding 
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could be sustained by the construction of new tests with prediction as to 
how they should behave factoually in new groups of differently selected 
subjects 

In one factorial study it was found that a primaiy factor was common 
to a set of tests that weie given by the projector method with individual 
tuning for each lesponse The interpretation of such a factor was uncertain 
Some psychological function might be involved m the projector tests which 
was absent fiom the other tests, but the explanation might also be that some 
motivational condition was common to the projector tests that was absent 
from the other tests and which would be of only incidental significance as 
far as the major pui poses were concerned 

Suppose that one of the examiners misundei stands the time limits for a 
set of tests and that he gives the shoiter time limits to a part of the group of 
subjects for some of the tests A factor might appear under certain circum¬ 
stances that would be incidental and of no fundamental significance, but the 
primaiy factors that aie significant might still be levealed An unexpected 
intenuption m a school examination, such as file drill, a street parade, or 
the anticipation of an impoitant school event, may act to introduce inci¬ 
dental faclois 

One of the most impoitant souices of incidental factors is to be found in 
the selective conditions If a group of subjects is selected because of quali¬ 
fication in a composite of two or moie tests, the unique variances of such 
selective tests combine to foim one or more incidental common factors 
which would have remained a part of the unique variance if the selective 
conditions had not been imposed The conelations between the factois are 
detenmned in laige part by the selective conditions If a group of subjects 
is selected because of certain test qualifications, it is to be expected that the 
primary factors will show correlations between factors that are different 
from the correlations between the same factois m an unselected population 
It must not be assumed that the factors are different just because they cor¬ 
relate differently in different populations This effect is well known with 
physical measurements—height and weight with intelligence, for example— 
whose mteicorrelations are determined in laige part by the selective condi¬ 
tions These changes do not affect the identity of the factors. An incidental 
factoi which is introduced by conditions of selection may be trivial or it may 
be significant, depending on the natuie of the unique variances which are 
intioduced into the common factors by the selective conditions 

It should bo remarked that m a well-planned factorial experiment the 
incidental factois are usually of secondary importance in comparison with 
the vaiiance that is assignable to the principal primary factors for which an 
experiment was planned When one or moie primaiy factois have relatively 
small variance and do not seem to lend themselves to clear interpretation, 
they should be repoited without mteipretation Some reader of such a re- 
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port may find a fruitful hypothesis for it, or the factor may be of only in¬ 
cidental significance, 

These few examples will serve to call attention to the fact that not all the 
pnmaiy factors can be expected to have meaning m the fundamental sense 
of representing functional unities whose identity transcends the particular 
occasion It does not follow that incidental factors are in any sense arti¬ 
facts They may represent genuine factors that were operating to produce 
the observed individual differences, but then significance may not extend 
beyond the particular occasion In that sense they are melevant to the 
puiposes of the experiments, even though they are valid as factors which 
can sometimes be identified 

An interesting application of second-order factois is an attempt to recon¬ 
cile thiee theones of intelligence, namely, Speaiman’s theory of a general 
intellective factor, Godfrey Thomson’s sampling tbeoiy, with what he 
calls "sub-pools”, and oui own theory of correlated multiple factors, 
which ai e interpreted as distinguishable cognitive functions The teti ad dif¬ 
ferences vanish when theie aie no pnmaiy factors common to the four tests 
of each teti ad, the coi relations being determined only by the general second- 
oidei factoi 



CHAPTER XIX 

THE EFFECTS OF SELECTION 


Types of selection 

Factorial results are affected by selection of subjects and by selection of 
tests It will be shown that the addition of one or more tests which are 
linear combinations of tests already in a battery causes the addition of one 
or more incidental factors If the given test battery reveals a simple struc¬ 
ture, the addition of tests which are linear combinations of the given tests 
leaves the structure unaffected, unless the number of incidental factors is 
so large that the common factois become indeterminate Such a situation 
violates the restrictive relation between the number of tests n and the num¬ 
ber of common factors r 

When a factor analysis has been completed for a test battery on one group 
of subjects, the administration of the same battery to a differently selected 
group of subjects will give a diffeient factorial result if the second group is se¬ 
lected by any criterion m the common factors If a simple structure is 
found for the first group of subjects, the same structuie will be found for the 
second group, but the correlations between the primary factors will be dif¬ 
ferent. Incidental factors are added to the common factors under certain 
conditions of selection of the subjects. It will be shown that, under these 
various conditions of selection of tests and subjects, a simple structure re¬ 
mains invariant It will also be shown that the physical interpretation of 
each primary factor lemams the same for wide vauations m selection of 
both subjects and tests but that their intercorrelations are altered by selec¬ 
tion of subjects. It can readily be seen that these considerations are of 
fundamental significance for factor analysis as a scientific method. 

These principles will be illustrated by numerical examples with the box 
problem The box dimensions, x, y, and z, may have low correlations m one 
group of boxes, but these corielations may be quite different if the analysis 
is repeated on another gioup of boxes so selected that they are all, say, rela¬ 
tively low and squatty Then the ^-variance is reduced, and the correlations 
may be altered Of course, these altered correlations of the primary factors 
do not deny the identity of the parameters, length, height, and width, for 
describing the box measurements in both box populations The factors are 
the same and their physical interpretations are the same m both popula¬ 
tions, even though their inteicorielations alter from one experimental popu¬ 
lation to another This simple principle will be illustrated m numerical ex¬ 
amples 
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Experimental dependence 

One of the most important souices of incidental factors which aie extiane- 
ous to the purposes of a factorial study is what we have called expei mental 
dependence In earlier studies with the factonal methods this source of fac¬ 
tors was not recognized, but foi a number of years this pnnciple has been 
applied in the preparation of test batteries in the Psychoinetnc Laboratoiy 
at the University of Chicago * In Table 1 we have the factoi pattern for a. 
set of five tests (1, 2, 3, 4, and 5) with two common factois, A and B The 
reduced correlation matnx of this battery would be of rank 2, corresponding 
to the number of common factors For each of these five tests there is also 
indicated in the factor pattern a unique variance Now suppose that the 
performance from which the score of test 1 was derived is also used to derive 


Table 1 



A 

B 


Unique Factors 


1 

X 

X 

X 



2 

X 

X 


X 


3 

X 

X 


X 
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X 

X 


X 


5 

X 

X 



X 

la 

X 

X 

X 




another score, la Then we have two test scoies derived fiom the same per¬ 
formance, and they are quite likely to shaie some of the uniqueness of that 
particular peifoimance This fact is indicated in the factor pattern by the 
two x’s in the third column Tests 1 and la share the same uniqueness In 
addition, the two tests 1 and la may also have unique factois that are not 
shaied, but these aie not indicated in the factor pattern of Table 1 

A simple example of this mannei of constiuctmgatest battery would be 
to lecord speed and accuracy as two scores fiom the same test performance 
in the attempt to covei all aspects of the domain that is being investigated 
The new factoi pattern, after adding test la, has thiee common factors 
The thud common factor has saturations in only 1 and la It should be 
noted that the lank of the reduced correlation nmtnx has been raised from 
2 to 3 by this manner of assembling the test batteiy In factoung the test 
battery it would be necessary to extiact one additional factor from the cor¬ 
relation matnx befoic the lesiduals vanish Aftei rotation of axes, a third 
common factor would be found on which only two tests have significant 
satui ations, namely, 1 and la Upon examination of their nature it would be 
found that these two tests donve fiom the same test perfoimance Such a 
factor on which two variant scoies of the same peiformance have satura- 

* The importance of experimental dependence in factor analysis was first pointed out 
by Ledyard Tucker 
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tions is called a doublet factor, and it is ignoied in interpretation. It rarely 
gives any leverage on the problem of interpreting what the factor might 
represent, except that it is something concerned with that test performance 
—a fact that was known before the factonng was begun For this reason it 
is best to avoid inserting in a test battery two or more measures that are 
taken from the same test peiformance 

If we push this effect to an extreme by adding a duplicate form of each 
test to the battery, or by using each test and a parallel form of it, then we 
have a batteiy of ten tests instead of the five tests with which we started 
If there is a unique variance for each test, then the factor pattern of Table 1 
is altered by introducing five new common factois so that there are seven 
common factors in all, the two factors common to the five original tests and 
the five unique variances which have now become common factors It is 
doubtful whether a useful resolution of the correlation matrix into common 
factors is then possible The unique variances have been made into common 
variances by the inclusion of a parallel foim of each test, and the common 
factors that might be significant would be obscured by the addition of five 
common factors that may make the factor problem insoluble 

It is good practice to inspect a test battery for experimental dependence 
before factoring is begun One of the most common forms of experimental 
dependence is that m which some kind of total score is added to a test bat¬ 
tery to be factored. A mental age, for example, is essentially a summation 
score. It is known before the analysis is begun that the total score will be 
represented by a test vector that is in the middle of the configuration and 
that hence it contributes nothing to the identification of the primary vec¬ 
tors which are at the corners or intersections of the configuration Nor does 
it contribute to the identification of a second-order general factor which is 
defined by the correlations of the primaries However, a total score may be 
found to have higher saturation on the second-order general factor than on 
any one of the primaries The best procedure is to investigate the structure 
of the domain without the summation score and to investigate the char¬ 
acteristics of the summation separately 

Linearly dependent tests 

The summation score is an example of both experimental dependence and 
linear dependence of scores If we add to a test battery a new test score 
which is merely some linear combination of test scoies fiom the battery, we 
have linear dependence which generally introduces a new common factor. 
An exception is that in which the tests that are combined have no unique 
variance—a situation that arises only larely The introduction of a new 
test which is a linear combination of tests already in the battery is represent¬ 
ed by the factor pattern of Table 2 Here we represent five tests (1, 2, 3, 4, 
and 5) with two common factors and five unique factois If a new test were 
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added by merely taking the sum of the test scores 1 and 2, we should have 
the test denoted (1+2) This test is indicated as paiticipatmg in the two 
common factors A and B and in the uniqueness which adds two more com¬ 
mon factors 

Univariate and multivariate selection of subjects 

When a factor study has been done on a particular group of subjects 
with a given test batteiy, it is of fundamental interest to know how the 
factorial lesults would be alteied if the analysis were repeated on another 
group of subjects differently selected Here we are not inteiested m whether 
one of the groups is in any way representative of a universe while the other 
group is not so representative For our purposes the two groups are co¬ 
ordinate, but they aie differently selected so that they differ m homogeneity 
in one or more factors The results for either group are assumed to be 
known, and we want to be able to predict the factorial i esults when we know 
how the second group was selected in comparison with the fiist group 


Table 2 



A 

B 

I 

2 3 

4 5 

1 

X 

X 

X 



2 

X 

X 


X 


3 

X 

X 


X 


4 

X 

X 



X 

5 

X 

X 



X 

( 1 + 2 ) 

X 

X 

X 

X 



Consider the selection of a second gioup of subjects so that they are moie 
homogeneous, or less, in one of the tests m the batteiy This test will be 
called the selection test, and it will be denoted l The variances of all the tests 
have been arbitrarily reduced to unity by standardizing the scores foi the 
first group so that in that group <r, = 1 and, m particular, so that trj = 1 Let 
the new group be selected so as to have a different standard deviation in the 
selection test l, and let it be denoted Sj+1 The new group of subjects will 
have different standard deviations m all the tests that have correlation with 
the selection test l. These new standard deviations will be denoted s. The 
problem then is to estimate the new test correlations and the new factonal 
results for the second group The selection of a new group on the basis of 
their homogeneity in one of the tests has been called univariate selection by 
Godfrey Thomson, who has investigated this problem and whose lesults 
will be summarized here Univariate selection can be either complete or 
partial If the new group of subjects is so selected that all the subjects have 
the same score in the selection test l (tri = 0), then we have complete univariate 
selection If the new group is selected so that o-;+0, then the selection will 
be called partial univariate selection as to the selection variable l It will be 
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shown that interesting differences in the factorial results are determined by 
univariate selection when <rj> 0 and when <n = 0, which represents complete 
homogeneity in the selection test Further, it will be shown that the fac¬ 
torial results are also affected by the eommunality of the selection test l, so 
that the results differ according to whether hf = 1 or h% < 1 

When the selection of a second group of experimental subjects is deter¬ 
mined by more than one selection test, we have multivariate selection. The 
principles of multivariate selection can be illustrated in terms of two selec¬ 
tion tests, denoted l and m Such a situation can be analyzed as two suc¬ 
cessive univariate selections In multivariate selection one is likely to be deal¬ 
ing with a composite selection test, c =/(Z,m) This is the case of multivari¬ 
ate selection that has been studied by Godfrey Thomson and by Walter 

L 


A 

Figure 1 

Ledermann Here, also, the factorial results are affected by complete and 
partial selection in the composite criteiion of selection and by the communal- 
lties of the selection variables. The theory of univariate and multivariate 
selection will be described and numerically illustrated, and a comparison 
will be made of the psychological interpretations of the resulting factors. 

Geometrical representation 

Before writing the vector equations which lepresent the effects of uni¬ 
variate selection on the standard deviations and on the test correlations, we 
.shall consider the geometrical interpretation of selection in terms of the test 
configuration In Figure 1 we have a set of six unit test vectors in a con¬ 
figuration of two dimensions At the start the problem will be simplified 
by letting all six of these test vectors be of unit eommunality in the space of 
the two common factors, so that the diagram represents the total test space 
as well The standard deviations of the tests are represented by the unit 
length of the test vectois, which coiresponds to the fact that a, — 1 for ail the 
tests The test corielations are represented by the scalar products of pairs 
of test vectors In n dimensions we should have a test configuration which 
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is spherical, in that the dispersions have all been normalized by reducing the 
scores to standard scores The correlations are unaffected by this standard¬ 
ization 

Now let another group of experimental subjects be selected so that they 
show less variability in the direction of a selection test, L The effect will be 
to alter the shape of the test configuration, so that it is narrower in the direc¬ 
tion of the selection test In fact, the configuration will have an ellipsoidal 
shape, so that the configuration of two dimensions in Figure 1 will be an 
ellipse determined by the termini of the test vectors Since the new group is 
selected so as to be more homogeneous m test L, it follows that the compo¬ 
nents in this direction of all the other tests will be affected proportionally. 
If the factor of test L is reduced in dispersion, the same factor is also reduced 
proportionally m all the other tests m which this factor enters as a compo¬ 
nent 

Figure 1 has been drawn to represent a reduction of one-half in the dis¬ 
persion of test L. The L-component of the other tests is then also reduced 
by one-half, so that the new test vectors are depressed in the direction paral¬ 
lel to test L, as shown in the diagram. The termini of the new test vectors 
now define an ellipse instead of the circle It should be noted that tests A 
and E are unaffected by the selection because these tests are orthogonal 
(uncorrelated) to the selection test 

The scalar product of the unit vectors L and D is the given correlation 
r Ln This correlation is also the cosine of the angle LOD, and it is the pro¬ 
jection of the vector D on the vector L This is the distance DX. It is the 
component of D in the direction of L. Reducing this component by one- 
half, we have the new test vector d as shown The length of this vector is 
the standard deviation of test D m the selected group The new vector for 
test L is l, which is colhnear with L but only half as long. The cross product 
Id is the covariance of tests L and D in the new group If each of these short¬ 
ened vectors, l and d, is extended to unit length, we have the vectors L' 
and D'. It should be noted that the vectors L and L' are identical, whereas 
D and D' are not the same vectors The correlation between two tests, such 
as C and D, is changed by the selection to a value which is the scalar prod¬ 
uct of the new vectors C' and D' 

The correlations after selection 

The effect of univariate selection on the test correlations can be shown 
vectonally In Figure 2 let L be a unit vector representing the selection 
test in the total test space Let / be a unit vector representing any other 
test m the battery. The components of J may be considered to be OX, 
which is orthogonal to L, and XJ, which is colhnear with L When selection 
is made on the basis of dispersion m L, the component OX of J will be un¬ 
affected, whereas the component XJ will be affected 
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Let the selection alter the dispeision of L so that its standard deviation 
is altered from <ti = 1 to si = p The test L is then lepresented by the test 
vector l, whose length is si — p. The new test veetoi l can be wntten in the 
form 

(1) l = h — th , 
where 

(2) t a 1 - p . 

The component XJ of J is the projection of J on L, and it is also the corre¬ 



lation r,i Since the effect of selection on the test vector J is propoitional to 
its component colhnear with L, we have 

(3) Jo — J — tr,iL j 

where Jo is the new vector for test J Its new standard deviation is the 
length of the vector J 0 Squanng (3), we have 

(4) Ji = (./ - trM* 
or 

(5) JI = = J 2 — 2 tr }l LJ + « 2 r 2 ,L 2 

The scalar pioduct JL is the correlation r } i and the scalai product L 2 = J 2 = 1. 
Hence 


( 6 ) 


Sj = 1 — 2 tr^ + i 2 r|, 
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which becomes 

(7) aj = l-rf,(2t-*). 

Foi convenience let g 2 = 1—p 2 From (2) we then have 

(8) 2t - f = q 2 , 
so that the new variance sf can be written as 

(9) s* = 1 - q 2 r 2 , , 
by which the new standard deviation becomes 

(10) s, = V1 — q 2 r 2 i • 

The numerical values of s, can then be computed, since the coirelations r,i 
and the constants p and q are assumed to be known 

By analogy with (3) we can write the corresponding vector equation for 
test K, and we then have 

( 11 ) Ko — K. — truL 

The new covariance c, k can be expressed as the scalai product «7&Eo, which 
by (3) and (11) is 

(12) c,k = JqKq = ( J — tr,iL){K — truL) ; 
and this becomes 

(13) c, k = JK — IruJL — tr,iKL + t 2 r, t r ulP- 

Recalling that the scalar products JK = r, k , JL = r,i, KL = r k i, and L 2 =l, 
we get 

(14) c, k = r 3 1 , — tr,ir k i — tr,ir k i + Pr,tru ■ 

Collecting terms, we obtain 

(15) c lk = r jk - r 3 iru{2t — f) , 

which, by (8), becomes 

(16) Cj k = r ]k — q 2 r,ir k i . 

This equation enables us to compute the new covariances to be expected by 
selection on test L 
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The new correlations can be obtained by the stretching factor, winch 
merely reduces the new scores to standard scores Vectonally, this stretching 
factor is represented by normalizing the new test vectois J 0 and IC a to unit 
length. Since the lengths of these vectors aie known to be their standard 
deviations as given by equation (10), we have foi the new correlations 


(17) 


a? jL 


SjS k 


or, in more complete form, 
(18) ,r ]k 


i'll. ~ tfi'iQ'ki 
SjSk 


where ,r,k denotes the test correlations after selection This is Godfrey 
Thomson’s equation for the new conclations aftei selection * In computing 
the new correlations, ,r, k , it should be leealled that r,, = r (I = 1, because the 
present problem is concerned with the total vananees of the tests It is not 
limited to the common factois The diagonals of the given correlations r lk 
are unity 

In general, the dimensionality of the test configui ation in the total test 
space is equal to the numbei of tests n When the test scoi es are normalized, 
each test vectoi is of unit length in the total test space, and the surface de¬ 
termined by the test vectoi teiimm is a spheie m n dimensions After selec¬ 
tion on one of the tests so as to l educe the vauance of the selection test L,we 
have an ellipsoid deteinuned by the test vector teimmi If the selection on 
test L is complete so that all subjects in the new group have the same scoie 
in test L, then we should expect to lose one dimension. The suiface deter¬ 
mined by the test vector teimmi would then be a sphere of dimensionality 
(n — 1) If n — 3, the spherical surface becomes a round pancake when p is 
small and a plane when p is zeio 

When s; = p = 0 so that 1 = 1, the selection on test L is complete The di¬ 
mensionality is then reduced to (n — 1), and equation (18) becomes 


(19) 


sTj k Tjk i 


r 3 k — rjir k i 

Vl - rh Vl - rh ’ 


* Godfrey H Thomson, “The Influence of Univariate Selection on the Factorial 
Analysis of Ability,” British Journal of Psychology (Gen Sec), Vol XXVIII (April, 
1938), Part IV, Walter Ledcimann, “Note on Piufcssor Godfiey FI Thomson’s Article 
‘The Influence of Univariate Selection on Factorial Analysis of Ability,’ ” British Jour¬ 
nal of Psychology (Gen See ), Vol XXIX (July, 1938), Part I, Godfiey FI Thomson and 
Walter Ledermann, “The Influence of Multivariate Selection on the Factorial Analysis 
of Ability,” British Journal of Psychology (Gen See ), Vol XXIX (January, 1939), 
Part III, and Godfrey H Thomson, The Factorial Analysis of Human Ability (New 
York: Houghton Mifflin Co , 1939), chaps xi and xn 
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which is the familiar partial correlation formula If the three-dimensional 
configuration has been represented on the surface of a sphere and if the selec¬ 
tion is complete on test L, the resulting two-dimensional configuration can 
be visualized by looking at the spherical model with test vector L m the 
line of regard The resulting two-dimensional configuration is then directly 
seen The partial correlations would then be represented by the scalar prod¬ 
ucts of pairs of these vectors after they have been normalized in the two 
dimensions that would be seen on a photograph of the sphere with test L at 
the center of the picture 

It has been pointed out by Godfrey Thomson that the well-known Otis- 
Kelley formula for the effect of restricted selection on the correlation co¬ 
efficient is a special case of equation (18) In this case s, = Sk because both 
variables are then assumed to be subject to the same restricted selection 
Equation (18) then becomes 


( 20 ) 


eTjk 


r,h — q^jiTici 


By (10) we have, when s } = s k , 

( 21 ) 1 - q 2 rji = 1 - rfri, , 


so that 


( 22 ) 


r,i — rki , 


and hence (20) becomes 
(23) 

From (9) we have 


aVjk 


_ r,k ~~ q 2 rh 


(24) 


_ 1 ,2 

qr }l — l s, 


so that (23) becomes 
(25) 

and hence 


a'f'jk 


r,k ~ (1 ~ s?) 


(26) 


.r } ksj = r jk — 1 + Sj , 


so that 

(27) 


s?(l — *r,k) = 1 — r } ic 
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from which we have 
(28) 


1 - r >k = „2 = 

1 - jr, » J <r?' 


which is Ihe Olis-Kelley formula These estimates of the effects of selection 
are, of couise, in the nature of expected aveiages, fiom which the selected 
samples would show landom deviations 

The communalities 

So fai we have consideied the effect of selection on the dispeisions and 
couelations which imply the total test space When the communalities are 
known in one coil elation matrix, the new communalities can be found by a 
slight adaptation of equation (18) The diagonal couelations aie then writ¬ 
ten = instead of unity Since the selection is assumed to be made by 
the test scoies, it follows that the uniqueness of the selection test, as well 
as its common-factot vauance, must be involved m the result 

Writing equation (18) foi the communalities, where r„ = /i 2 , we have 


(29) 


ellj — 


ft? ~ f r fl 

1- 5*r?j 


0^0. 


wheic the denominator is written by equation (10) It should be noted that 
equation (29) applies to the computation of all the communalities, except for 
the selection test Hence the lestuction foi equation (29) that 
Whenj = Z, we can wiite this equation in the modified foim 


(30) 


a? II sft( 


ft? - <7 2 • ru • hi 


1 - q 2 r u hf ’ 

where one of the correlations ru is unity and the othei is hf Then 


(31) 


= hj( 1 - g 2 ) ft?P 2 
‘ 1 1 - q 2 hf 1 - (f/i? ’ 


by which the new communahty of the selection test can be computed Equa¬ 
tion (31) may be denved vectorially as follows 

Lot L in Figure 3 be the selection test vectoi in the total test space, and 
let L r and L u be the common-factor component and the unique component 
of L, so that L = L 0 -\-L u , By selection the configuration of test vectors is 
Hist contracted by the propoition p m the diiection of L and then noirnal- 
ized Contracting L to pL and then normalizing leproduces L again Hence 
the selection vector L is not altered by selection, except, of course, when 
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p = 0, in which case pi becomes a null vector and the configuration is reduced 
to dimensionality (r—1) 

The length of the common factor component L c is hi, and the projection 
of L c on L is pL, whose length is h\ The correction vector for L c , which is 
introduced by selection, is therefore —th\ L, as shown in Figure S The al¬ 
tered vector Lc can now be wntten as 

(32) ,L'c = L c - th\L , 

where S L' C repiesents the distortion of L c that is introduced by compressing 
the configuration m the direction of L befoie the vcctois are finally normal¬ 
ized again 



In order to find the length of ,L' we write 

(33) ( ,L' c y — LI — 2thfL„L + i 2 hjL 2 . 

Since L 2 = 1, L 2 = hi , and, since the scalar product L c L = h l , equation (33) 
becomes 

(34) {,Ly =* A? - 2th\ + t*h\ , 
which, by (8), reduces to 

(35) CL:) 2 = Af( 1 - q*hl) . 
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The scalar product of e L' c and L in Figure S can be written either as a 
product from equation (32) or as the product of the lengths of L and e L' 
and the cosine of their an gular separation The length of L is unity, the 
length of ,L’ C is /i ( vT —§ 2 /if; and, since the cosine of the angular separation 
is also the projection of L on the new common-factoi space, namely, s hi, we 
have 

(36) ,L' C L = ,hi ■ A,V'l - qVi$. 

The same scalai piodurt can be wntten by equation (32) 

(37) ,L’ C L = LL C - thfL 2 , 
which becomes 


(38) ,L' C L = h f - th\ 
or 

(39) ,L[L = h\p . 
Equating (36) and (39), wc get 

(40) hip = JuhiV 1 — qV i? , 


and writing this explicitly for the new communahty of the selection test, 
we have 


(31) 


.M = 


hip 2 

1 - q 2 h\ ’ 


as previously written By this equation the new communahty of the selec¬ 
tion test can be computed 

In solving a complete problem of this kind it is convenient to use God¬ 
frey Thomson’s equation (18) for all the new conelations, including the 
communalities, except that of the selection test, foi which equation (31) 
applies 

Computational sequence 

The pioblem of detei mining the effects of univaiiato selection starts with 
the correlation matrix and the communalities, as well as the degree of selec¬ 
tion on the selection test, which is denoted L with subscnpt 1 It is assumed 
that the given scoies have been standardized so that a, = 1 for all tests The 
selection of a new group is such that the standard deviation of the selection 
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test is changed from the given value <r ; = 1 to a new value, s ( , which is known 
The ratio of the standaid deviationa is 


Si 

Cl 


Si = V 


wheie p is a parameter that determines the new factonal lesults The fol¬ 
lowing convenient constants are then computed p 2 , q 2 = 1 — p 2 , and q 
The given correlation matnx is written with unity in the diagonals 
Equation (18) can be written in the form 


(41) R, = MCM = M(R - UU')M , 


where R a denotes the expected correlation matrix after univariate selection 
The matrix U is a column vector with elements u, = qr ]h and the matrix M 

Table S 

Given Correlation Matnx R., 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

l 

1 000 

000 

000 

350 

606 

350 

606 

000 

000 

404 

2 

000 

1 000 

000 

606 

350 

000 

000 

350 

606 

404 

3 

000 

000 

1 000 

000 

000 

606 

350 

606 

350 

404 

4 

.350 

606 

000 

1 000 

606 

175 

303 

303 

525 

552 

5 

006 

350 

000 

606 

1 000 

303 

525 

175 

303 

552 

6 

350 

000 

606 

175 

303 

1 000 

606 

525 

303 

552 

7 

606 

000 

350 

303 

525 

606 

1 000 

303 

175 

552 

8 

000 

350 

606 

303 

175 

525 

303 

1 000 

606 

552 

9 

000 

606 

350 

525 

303 

303 

175 

606 

1 000 

552 

10 

404 

404 

404 

552 

552 

552 

552 

552 

552 

1 000 


is a diagonal matrix with elements d,= 1/s, Equation (41) indicates the 
computational order One may compute, fust, the column vector U, then 
the covariance matrix C = XJU' of order nX», and, finally, R, The correla¬ 
tion matrix R s is, of course, symmetric, and it also has umty m the diag¬ 
onals 

When the correlation matrix R, has been computed, the communalities 
may be computed separately by equation (36), wheie it must be remembered 
that ru is written as hi 

Examples of univariate selection 

The principles of the preceding sections will be lllustiated by a numerical 
example with a physical model A variant of the box problem will be used 
The factor matrix of Table 4 represents ten measures of the box problem, in¬ 
cluding the basic parameters, height, width, and length, and such other box 
measurements as the diagonal of each face, the area of each face, and one 
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complex measure, such as the volume In order to illustrate the effects of 
selection, the given example m Table 4- is arranged to represent the situation 
in which the thiee basic parameters aie uncorrelated Further, m order to 
show the effect of selection on the communahties, these have been arranged 
in the factoi matrix as markedly below unity, namely, 70 foi all ten meas¬ 
ures The corresponding correlation matux is shown m Table 3 These are 
the given data 

Table 4 

Given Factor Matrix Fo 



I 

II 

III 

1 

265 

- 270 

546 

2 

542 

632 

047 

3 

542 

- 424 

- 473 

4 

620 

448 

313 

5 

566 

116 

555 

C. 

620 

- 519 

- 165 

7 

566 

- 514 

257 

8 

740 

- 051 

- 386 

9 

740 

336 

- 195 

10 

815 

- 024 

047 


Table 5 

Computation of s> 



r )l 

(ir 3 i 

Ah 

S i 



"h 


1 

1 000 

800 

640 

300 


600 

i 

667 

/<= 60 

2 

000 

000 

000 

1 000 

1 

000 

i 

000 

;j ! = 36 

3 

000 

000 

000 

1 000 

1 

000 

i 

000 

r/ 2 = 64 

4 

350 

280 

078 

922 


960 

i 

042 

< 7= 80 

5 

606 

485 

235 

765 


875 

i 

144 

/= 40 

G 

350 

280 

078 

922 


960 

i 

042 

(«= 16 

7 

606 

485 

235 

765 


875 

i 

144 

S 2 = 1 — C 2 ! 2 

8 

000 

000 

000 

1 000 

1 

000 

i 

000 

(f= 1 — p 1 

9 

000 

000 

000 

1 000 

1 

000 

i 

000 

t = 1 — p 

10 

404 

323 

105 

896 


946 

i 

057 

u,=</r,i 










m, = 1/s, 


It will now be supposed that a new collection of boxes is measuied which 
is moie homogeneous in the measurement of test 1, so that its standaid de¬ 
viation is 60 in the new population instead of unity, as in the given popu¬ 
lation The problem is now to investigate the effects of such selection on the 
coirelations and on the simple stiucture The reduced standaid deviation 
is the parameter p of Table 5, and from this value aie determined the other 
parameters, such as q, t, and the new standard deviations s, of the othci 
measutes that are con elated with test 1 These are all listed m Table 5 
In the same table we have the detailed computations for the new standai d 
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deviations s } and the reciprocal rn, The computations of this table are de- 
tei mined by equations (9) and (10) 

The new covatiances after selection are determined by equation (16), 
which can also be written m matux foim, as shown m Table 6 The entries 
in this table are the cross products of the new scoies, whose standard devia¬ 
tions have been depressed Hence these entries are not coi relation coeffi¬ 
cients The diagonal entries are the new variances of the tests The corre- 


Table 6 


Governance Mali ix C a = Ri — U0' or c B = r,k — u,Uk 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

360 

000 

000 

126 

218 

126 

218 

ooo 

ooo 

146 

2 

000 

1 000 

000 

606 

350 

000 

000 

350 

606 

404 

3 

000 , 

000 

1 ooo 

000 

000 

006 

350 

606 

350 

404 

4 

126 

606 

000 

922 

470 

097 

167 

303 

525 

462 

5 

218 

350 

ooo. 

470 

765 

167 

290 

175 

303 

395 

6 

120 

000 

606 

097 

107 

922 

470 

525 

303 

462 

7 

218 

ooo 

350 

167 

290 

470 

765 

303 

175 

395 

8 

000 

350 

606 

303 

175 

525 

303 

1 000 

606 

552 

0 

000 

600 

350 

526 

303 

303 

175 

606 

1 000 

552 

10 

146 

404 

404 

462 

395 

462 

395 

552 

552 

896 


Table 7 

New Correlation Malrtx R» = MC.M 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

1 000 

ooo 

ooo 

219 

416 

219 

416 

ooo 

ooo 

256 

2 

ooo 

1 ooo 

ooo 

632 

400 

000 

000 

350 

606 

427 

3 

ooo 

ooo 

1 ooo 

000 

000 

632 

400 

606 

350 

427 

4 

219 

632 

ooo 

1 000 

560 

105 

199 

316 

547 

508 

5 

416 

400 

ooo 

560 

1 000 

199 

379 

200 

347 

478 

6 

219 

000 

632 

105 

199 

1 000 

560 

547 

316 

508 

7 

416 

000 

400 

199 

379 

560 

1 000 

347 

200 

478 

8 

000 

350 

606 

316 

200 

547 

347 

1 000 

606 

583 

9 

000 

606 

350 

547 

347 

316 

200 

606 

1 000 

583 

10 

250 

427 

427 

508 

478 

508 

478 

583 

583 

1 000 


spondmg correlation coefficients are shown in Table 7, and these repiesent 
Godfrey Thomson’s equation (18) with unit diagonals. They can be ob¬ 
tained from the covanances of Table 6 by the stretching factors m, = 1/s,, 
which are the elements of the diagonal matrix M This matrix is identical 
with the table of covariances, except for the fact that the standard devia¬ 
tions have been stretched to unity 

The new communalities are determined by the detailed calculations 
shown m Table 8 from (36), in which r (! is wntten as hi Finally, when the 
new correlation matux with communalities m the diagonal cells is factored, 
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we get the new factor matrix F a , as shown in Table 9. Here the number of 
factois is again three, as m the given eonelation matrix, thus verifying God¬ 
frey Thomson's theorem that the rank of the correlation matrix with com- 
munalities lemams unaltered with univariate selection However, an ex¬ 
ception should be mentioned In complete or total univanate selection on a 
factor or its equivalent—a test with perfect communahty—-the new stand¬ 
ard deviation of the selection test 01 factor is zeio, and the rank of the new 
correlation matnx has then been reduced by 1 In the piesent example the 
rank of the new correlation matrix, aftei complete selection by a factoi oi 
by a perfect test, is reduced to 2 Several examples of this type will also be 
shown 

Perhaps the most important consideration in this problem is to asceitam 
what happens to the identification of the pnmai y factors In Figure 4 we 


Table 8 


New Communalihes 


„h; = 


h! 


■ 


1 - q*r?l 


Table 9 

New Factor Matrix F s 



h 

r,i 

r »l 

qVJi 

Num 

Denora 

.'■? 


I 

11 

III 

1 

700 

837 

700 

448 

262 

552 

457 

1 

334 

- 340 

688 

2 

700 

000 

000 

000 

700 

1 000 

700 

2 

544 

634 

048 

3 

700 

000 

000 

000 

700 

1 000 

700 

3 

543 

- 425 

- 474 

4 

700 

126 

016 

010 

690 

990 

697 

4 

638 

379 

385 

5 

700 

218 

048 

031 

670 

970 

691 

5 

561 

023 

619 

b 

700 

120 

016 

010 

690 

990 

697 

6 

637 

- 538 

- 067 

7 

700 

218 

048 

031 

670 

970 

691 

7 

561 

- 507 

359 

8 

700 

000 

000 

000 

700 

1 ooo 

700 

8 

742 

- 051 

- 387 

9 

700 

000 

000 

000 

700 

1 000 

700 

9 

742 

337 

- 196 

10 

700 

146 

021 

014 

686 

986 

696 

10 

820 

- 075 

151 


have the configuration of test vectors befoie and after selection on test 1. 
In this diagram the test vectois are represented by the method of extended 
vectors, so that the tluee-dunensional configuiation can be seen in a plane 
The given positions of the test vectors are denoted a, and the new positions 
by b A glance at this figure shows that the position of the selection vari¬ 
able 1 lemams unaltered So do also the tests which are uncorrelated with 
the selection test These are tests 2, 8, 8, and 9 These remain unalteied 
The other test vectors— 6, 6, 7, and 10 —-move in a direction radially 
fiom the selection vector, so that the tnangular configuiation is still re¬ 
tained In the piesent example the actual dimensions of the triangle even 
remain unalteied We sec, then, that m this example of partial univariate 
selection the simple structure remains invariant If two investigators start¬ 
ed with the two eonelation matnces, one with the first group of objects and 
the othei with the specially selected gioup of objects, they would arnve at 
the same simple structuie, and they would identify the same primary far- 
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tora or parameters But their results would differ as to the correlations be¬ 
tween the measurements with which they started In the present example 
the correlations between the primary factors would remain the same for the 
two groups of objects Variations of these principles will be shown in addi¬ 
tional examples 



The most important principle to be drawn from analyzing the effects of 
selection on factorial results is that under wide variations in the conditions 
of selection the simple structure is invariant, so that the primary factors or 
parameters are the same The correlations of primary factors are altered 
from one selected group to another A simple example will serve to illus¬ 
trate this principle, which is well known m other contexts The correlations 
between height, weight, and intelligence can be made to take widely differ¬ 
ent values, depending on the selection of the experimental gioup If chil- 
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dren of all ages and statures aie included, then the correlation between 
stature and intelligence will be found to be appreciable This correlation is 
not spurious It is collect to say that there actually is a high correlation 
between intelligence and stature for any expenmental population of chil¬ 
dren with a wide lange in chronological age The taller children are gen¬ 
erally older, and they scoic higher on the tests If the experimental popu¬ 
lation is limited to point age, the con elation between statuie and intelli¬ 
gence vanishes, 01 neatly so The mteipictation is not to deny the meaning 
of the vaiiables or the factois These aie the same measmements of stature 
and test scoie for the two gioups It is the coirclations between the factors 
that aie alteied by the selective conditions In the box example it will be 
seen that the correlations between height, width, and length of the boxes 
can be alteied by the selection of each collection of boxes, but the physical 
interpretation of the tliiee parameteis remains precisely the same in the 
seveial box collections We dwell on this point in detail because it is the 
souice of misunderstanding of the factonal methods, in that some cntics aie 
inclined to deny the validity of the physical interpretation of factois merely 
because their con elations are alteied by conditions of selection N 

We turn now to several othei examples of univariate selection with the 
same box pioblem The detailed computations will not be repeated because 
the pnnciples weic illustrated with the fust numeiieal example The next 
few examples have been pieparecl to show the effects of univariate selection 
of factois, as distinguished fiom selection by tests The Ihiee primary fac¬ 
tois for the box problem may be denoted Pi, P 2 , and P 3 These may be re¬ 
garded as unit vectois determined by the first three measuies, 1, 2, and 3, 
in the battei y Figure 5 has been diawn to show the effects of univauate se¬ 
lection on the factor Pi In this set of diagrams —Figures 5-10, inclusive— 
the selection variable is denoted S The method of extended vectois has 
been used to show the entire three-dimensional configuration 111 a single 
diagram in the plane of the papei The given positions of the test vectors aie 
shown by their numerical identification— 1, 2, 3, etc The now positions 

* Godfiey Thomson, who has contributed fundamental theory on the problem of uni- 
vaiiate and multivariate selection in relation to factor analysis, has expressed doubt as 
to whether the primary factors can be intei preted as basic and identifiable psychological 
processes His reason is mainly that the conelations between the primary factois of a 
simple stiuctuie aie altered and determined in part by the conditions of selection and, 
blither, that incidental factors can be added by certain conditions of multivanate selec¬ 
tion Our inteipretation is that the primary factois repiesent the same basic piocesses in 
different conditions of selection and that it is the conelations between these parameters 
that are alteied rather than the fundamental meaning of the parameters themselves. 
The incidental factors can be classed with the residual factois, which leflect the condi¬ 
tions of paitiouUr expenments These extraneous factors do not show the invariant 
chaiacteristics that have been shown foi the more basic pnmaiy factois In presenting 
Godfrey Thomson’s theoretical work on the selection problem, we are giving a less pessi¬ 
mistic interpretation of the factorial results 
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are denoted lx, £x, 3x, etc This set of six diagrams was designed by Led- 
yard Tucker to show graphically the effects of vanous conditions of uni¬ 
variate selection on the test configuration 

Figure 5 shows the effect of partial selection on the factor P. Note that 
the position of the selection variable S = l=lx remains unaltered The po¬ 
sitions of the test vectors £, 3, 8, and 0 remain the same because they are 
uncorrelated with the factor P i The othei test vectois move radially from 
the selection vanable S, as shown m the triangular configuration The sim¬ 
ple structure is unaltered In the same figure we have a diagiam showing 
the effect of total selection on P L This means that all the boxes in the new 
box population have the same height The individual box shapes will be 
determined by only two parameters, since one of the parameters has be¬ 
come a constant and does not affect the individual diffeienoes among the 



objects Theiefore, we expect the factorial result after selection to be of 
rank 2 This is verified m the diagram, which was plotted after making the 
computations that have been explained The variable 1 disappears from 
the second analysis because it is a null vector This is what happens occa¬ 
sionally when the communality of a variable turns out to be vanishingly 
small It usually happens when the coi responding coi relations are also 
small The new configuration lies entirely m the base line of the figure, 
which repiesents only two dimensions. 

If an investigator were to make a factor analysis of such a collection of 
boxes, he would find only two factors If he weie to plot the resulting con¬ 
figuration, he would get a simple structure, as shown m Figure 6 Here we 
see that, even though one of the three factors has been entirely eliminated 
from the individual differences and from the analysis, the same simple 
structure is identified for the remaining two factois The interpretation of 
the two primary factors, width and length of boxes, is the same m the two 
box populations. 
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Figure 7 was drawn to show the effect of univariate selection on the com¬ 
posite variable (P2+P3) The first diagram shows the effect of partial selec¬ 
tion on this composite variable, which is denoted S The given configura¬ 
tion of ten vectors is shown on the inside triangle, which is identical with 
that of Figure 5 The effect of selection is to move the termini of the test 
vectors radially fiom the selection vector to the new positions, denoted lx , 
Sx, Sx, etc The vanablc 1 = lx is unaltered because it is orthogonal to the 
composite variable S The others move as shown by the radial lines from S 
The new configuration shows the same simple structure as before, with the 
same primary factois in the tests 1, 2, 3 Hence the interpretation of the 
primary factors would be the same as before selection The correlations be¬ 
tween the primary factors have been altered 



Total Selection on /f 


Figure 6 

In the second diagiam of Figure 7 we have the effect of total selection on 
the composite variable S— {P 2 +P 3 ) Here we start, as before, with the 
given triangular configuration of ten test vectois extended to the tangent 
plane. The new configuiation, after selection on S, is of two dimensions, as 
is to be expected after complete selection on a factor The new configuration 
is indicated along the upper horizontal line of the diagram Tests 2x and 
9x are orthogonal to test 1 , and so are also tests Sx and 8 x If an investi¬ 
gator were to start with the selected box population and proceed with a 
factoi analysis, he would find only two factors in this case The configura¬ 
tion that he would find is shown in Figure 8 , wheie it will be seen that 
the simple stiuctuie is again the same for the two factors that remained 
in the system The primaly factois aie again identified by test 1 and the 
combination of tests 2 and 8 The physical mterpietation of the factor Pi 
would be the same as before The intei pretation of factors P 2 and P 3 would 
be obscuied because the investigator would supposedly not know that he 
was dealing with a fieak collection of boxes, which had been so selected that 
the sum of the width and length of each box was the same for all In the 
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first diagram, of Figui e 7 it is seen that with partial selection this ambiguity 
does not arise 

We turn next to an even more stringent selective condition, namely, the 
composite variable S = (P1+-P2+P3) This is the sum of the three box di¬ 
mensions In Figure 9 the inside triangle shows the given configuration as 
before The composite variable S is m the middle of the configuration at 
test 10 The new configuration, after partial selection on the composite 
vanable S, is shown by the outside triangular configuration The new test 
vectors all lie in the outside triangle, and then locations ai e determined by 
the radial lines from the selection variable as before It can be seen that the 
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simple structure is here again retained aftei selection and that the primary 
factors are the same, namely, those which are determined by tests 1 , 2, and 
3 Hence we conclude that partial selection on a linear combination of even 
all the factors leaves the simple structure, as well as the physical interpreta¬ 
tion of the primary factors, invariant 

However, if the boxes are assembled so as to satisfy the conditions of total 
selection on the composite variable S, then the lank is reduced to 2, and m 
Figure 10 we have the plot that an investigator would make with the new 
configuration Here it is seen that the simple stiucture has been destroyed, 
so that the pnmaiy factors are no longei identifiable We conclude, there¬ 
fore, that conditions of selection can be made so extreme that the simple 
structure is destroyed In such a population the primary factors cannot be 
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determined In this case we should be dealing with a set of boxes so selected 
that the sum of the three box dimensions was the same for every box in the 
collection For partial selection, as shown in Figure 9, the simple structure 
remains invariant Hence we conclude that it is the total selection on this 
composite variable which destroys the simple stiucture 

Multivariate selection of subjects 

So far it has been assumed that the selection of subjects is determined by 
their homogeneity in a single criterion variable, even though this variable 
is itself a linear combination of seveial factois Selection of subjects can be 
made to depend on seveial variables, with the lestriction that the mtercor- 
lelations of these vaiiables must have certain prescribed values In these 
cases it may not be possible to describe the correlations m the selected group 
as an alteration in the homogeneity of a single variable, not even a composite 
selection cntenon We then have multivariate selection of subjects However, 
multivariate selection can be described in terms of successive univariate se¬ 
lection by the formulae alieady discussed, provided that the successive vari¬ 
ables are properly chosen This procedure will be illustrated by a numerical 
example that has been used by Godfrey Thomson in his discussions of multi¬ 
variate Selection + 

In analyzing multivariate selection it is convenient to divide the van- 
abLes into two groups, namely, those which aie directly involved in the selec¬ 
tion and those which are not directly involved The foimer, by which the 
selection is determined, will be denoted by subscript j and the others will be 
denoted by subscript k, The given correlation matrix can then be sectioned 
in the following manner 


R„ 

Rik 

Rn 

Rkk 


The correlations of the selection tests aie shown m section Ii„ befoie selec¬ 
tion The corielations of the same tests aftei selection may be symbolized 
by the matrix V n , which shows the new coirelations that aie imposed by 
selection The new con elation matrix can then be symbolized in sectioned 
form as follows 


v u 

v tt 

V, 

V n 


* The Factorial Analysis of Human Ability, p 187 
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which represents the correlations to be expected after selection The diag¬ 
onal elements of V ,, show the new variances of the selection tests, and the 
side entries of V :] show the covariances to be imposed by selection 

The problem is, then, to determine the expected correlations in the sec¬ 
tions V k k and V, k , which is the transpose of V k3 A formal matrix solution 
to this problem has been written by A. C. Aitken and reported by Godfrey 
Thomson * The solution is as follows 


(42) TV = VjjRj'Rjk , 

(43) Vki = Rkk - Rk^R- 1 - R-^VjjR* 1 ) R ]k , 

(44) TV = V' lk 


Godfrey Thomson’s numerical example will be used for the present dis¬ 
cussion to illustrate the analysis of multivariate selection in teims of suc- 


Table 11 


Table 10 

Given Correlation Matrix 



L 

M 

1 

2 

3 

4 

5 

6 

L 

1 0000 

0000 

3741 

- 3741 

0935 

0802 

0668 

0534 

M 

0000 1 

0000 

9274 

9274 

6416 

5500 

4583 

3667 

1 

3741 

9274 ] 

L 00 

72 

63 

54 

45 

36 

2 

- 3741 

9274 

72 

1 00 

56 

48 

40 

32 

3 

0935 

6416 

63 

56 

1 00 

42 

35 

28 

4 

0802 

5500 

54 

48 

42 

1 00 

30 

24 

5 

0668 

4583 

45 

40 

35 

30 

1 00 

20 

6 

0534 

3667 

36 

32 

,28 

24 

20 

1 00 


Given, 

Factor 

Matrix 



I 

1 

9 

2 

8 

3 

7 

4 

6 

5 

5 

6 

4 


cessive univariate selection His numencal example is repioduced in the 
correlations between the six numbered variables of Table 10 The columns 
L and M have been added, and they will be described presently The given 
correlation matrix is of order 6X6, and it is of unit rank The corresponding 
one-column factor matrix is shown in Table 11 In older to illustrate multi¬ 
variate selection, Thomson imposes a change in the vanances of tests 1 and 
2 and also in the covariance Cu The given section of the correlation matrix 
for these two variables is 


1 

Rji — 

2 


1 2 


1 00 

72 

72 

1 00 


* Ibid , p 189 
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where the unit diagonals represent the unit variances of the tests at the 
start and where the given correlation r 12 = .72 This section of the correla¬ 
tion matrix is to be changed by selection to 



where the two test variances are depressed to 36 and the covariance to 
C 12 — 30 The pioblem is now to determine the expected covariances m the 


M 



Figure 11 


||git of the 6X6 matux and then to factor the coiicspondmg conelation 
l^iitrix. The direct method of Aitken can be used to solve the problem It 
also be represented as a form of successive univanate selection 
In Ftgui ell the two unit vectors, T x and T 2 , represent the two tests 1 and 
12: They have been drawn so that the cosine of the angular separation is 
,72, which is the given correlation ru. The relation between these two tests 
which is imposed by selection is shown by the two vectors T[ and T' 2 , with 
lengths of .60 and scalar product of 30 as required by the new matrix V,, 
The two configurations have been an anged m Figure 11 to utilize the sym¬ 
metry in this case in order to simplify the transformation from one to the 
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other The new 01 thogonal selection variables ai e di awn in the figui e, name¬ 
ly, L and M 

The transformation fiom the given configuration to the new configuia- 
tion can be expressed m terms of two successive umvanate selections, first 
on M and then on L The fust selection on M is specified by the restriction 
that 


5744 

.9274 


6193 , 


so that the new vectors become (Ti—A) and (Tt—A). Then follows a uni¬ 
variate selection on L, with the value 


1732 
P 3741 


4630 , 


so that the new vectors become 


(45) 

T[ = (Ti — A — B) , 

(46) 

T', = (T 2 - A+B) , 

which define the new configuration and the covanances 

The first step is to add two lows and columns to the given coirelation 
matrix foi L and M in Table 10 The unit vector M bisects the angle be¬ 
tween T i and T 2 , so that we can write 

(47) 

M = Ci{T t + r.). 

Hence 


(48) 

? jm = C L (X3 1 “f“ ^ji) ) 

wheie 


(49) 

1 

v 2(1 + r 12 ) 


In the same manner the unit vector L is a linear combination of T j and T«, 
orthogonal to M, so that 

(50) L — C2(Ti — T{) , 


and hence 
(51) 


r,i = c 2 (r,i — r ]2 ) , 
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where 

(52) 


Ci = 


1 _ 

V2(l - ru) 


Having thus determined the correlations of the six tests with L and M m 
Table 11, we determine the new covariances after selection on M These co- 
variances are shown in Table 12 The covariances after further selection on 
L are then computed in the same manner, and these are listed m Table IS 
It is not necessary here to express Table 12 m the form of correlation co¬ 
efficients 

Table 18 

Covariance Matrix after Selection on M 



L 

M 

1 

2 

3 

4 

5 

6 

L 

1 0000 

0000 

3741 - 

- 3741 

0935 

0802 

0668 

0534 

M 

0000 

3835 

3557 

3567 

2460 

2109 

1767 

1406 

1 

3741 

3557 

4699 

1899 

2032 

2255 

1880 

1504 

2 

- 3741 

3557 

1899 

4609 

1932 

1655 

1380 

1104 

3 

0935 

2400 

2632 

1932 

7462 

2024 

1687 

1350 

4 

0802 

2109 

2255 

1655 

2024 

8135 

1446 

1157 

5 

0668 

1757 

1880 

1380 

1687 

1446 

8705 

0964 

6 

0534 

1400 

1504 

1104 

1350 

1157 

0964 

9171 


P = 


5744 

9274 


« 6193 


p s = 3835 
q 2 = 6165 
q = 7852 


Table 13 

Covariance Matrix after Selection on L 



L 

M 

1 

2 

3 

4 

5 

0 

L 

2145 

0000 

0802 - 

- 0802 

0200 

0172 

0143 

0115 

M 

0000 

3835 

3557 

3557 

2460 

2109 

1757 

1406 

1 

0802 

3557 

3599 

2999 

2357 

2019 

1684 

1347 

2 

- 0802 

3557 

2999 

3599 

2207 

1891 

1576 

1261 

3 

0200 

2460 

2357 

2207 

7393 

1965 

1638 

1311 

4 

0172 

2109 

2019 

1891 

1965 

8084 

1404 

1123 

5 

0143 

1757 

1684 

1576 

1638 

1404 

8670 

0936 

6 

0115 

1406 

1347 

1261 

1311 

1123 

0936 

9149 


1732 

v 3741 _ 

2144 
7856 
q = 8863 


4630 


In Table 14 are recorded the correlation coefficients corresponding to the 
covariances of Table 13, and m Table 15 we have the final factor matrix 
after selection, which is also represented in Figure 12 It will be seen that 
the final factor matrix is of unit rank, except foi the doublet factoi m the se¬ 
lection tests 1 and 2 The nature of this doublet factor is explained in Ta¬ 
ble 2 at the beginning of this chapter The additional test is there a linear 
combination of the first two tests of the battery Here the same effect is 
shown because the two unique factors have been inti oduced into the com¬ 
mon factors to become an incidental common factor The multivariate se¬ 
lection with the restriction that the new correlation rn shall have an arbi- 
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trarily chosen value of ri 2 = 83 mtioduees an incidental factor, namely, the 
doublet factor II m Table 15 This introduction of an incidental factor into 
the common factois has not disturbed the single common factor, I, with 
which the problem started in Tables 10 and 11 The saturations of the fac¬ 
tor I have been altered, but not its identity. 

Summary 

Since factoi analysis starts with a set of measurements foi each individual 
member of an expenmental group, it is evidently of fundamental significance 


Table H Table 15 

New Correlation Matrix New Factor Matrix 



1 

2 

3 

4 

5 

6 


r 

II 

1 

87 

83 

46 

38 

30 

24 

1 

82 

45 

2 

83 

78 

43 

35 

28 

22 

2 

76 

45 

3 

4fi 

43 

31 

26 

21 

16 

3 

56 

0 

4 

38 

35 

26 

21 

17 

13 

4 

46 

0 

5 

30 

28 

21 

17 

14 

11 

5 

37 

0 

6 

) 24 

22 

16 

13 

11 

08 

6 

29 

0 


JT 



Figure 12 


to know how the factorial lesults aie affected by the manner in which the 
experimental individuals are selected To the extent that a simple structure 
and its associated primary factors are invariant under changes in selective 
conditions for the experimental group, we can have confidence that the 
primary factors represent identifiable processes whose nature transcends the 
circumstances under which the experimental subjects happen to be selected 
It does not follow that these identifiable and interrelated processes are 
therefore unique m the mathematical sense of being the only set of parame¬ 
ters that can be used for describing the dynamic system thatpioduces the 
observed individual differences. The invariance of simple structure under 
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changes m selective conditions does imply, however, that the processes so 
identified aie likely to be fruitful landmarks in a logical description of the 
system. 

By univariate selection is meant the selection of a gioup of experimental 
subjects so that their standard deviation in a selection variable shall be dif- 
feient fiom unity, which is the dispersion m a set of normalized scores for an 
initial gi oup of subj ects The criterion 01 selection variable may be defined 
by one of the tests m the battery, or it may be a linear combination of sev- 
eial tests or factors of the initial analysis If the selection cutenon is corre¬ 
lated with any of the tests in the battery, then there will be an effect on the 
couelations between the tests in the selected group of subjects The ex¬ 
pected correlations in the new group can be computed by Godfrey Thom¬ 
son’s equation (18) The new eommunahtics can be detei mined m an 
analogous mannei 

When the new group of subjects is assembled according to dispelBion m 
a selection variable L, so that its standaid deviation Si satisfies the in¬ 
equality 0 l, then the new group is assembled by partial selection in the 
sense that vanance in the selection vauable is not entirely excluded; hut 
it is diffoient from that of the initial gioup of subjects AVhen the new gioup 
is assembled so that its dispeision m the selection variable vanishes entne- 
ly, then its standaid deviation Si — 0, and the new group is then desenbed by 
complete or total selection on the selection variable 

The rank of the reduced con elation matnx for the new gioup is the same 
as for the initial gioup of subjects, except when the selection is complete 
on the selection variable, in which case the dimensionality is l educed by 1 
We then have fiist-order partial conelation coefficients, with the selection 
vauable constant, and the lank r is then reduced to (r — 1) 

In paitial selection when the lank of the reduced correlation matrix is 
invaiiant, the simple structure is also invariant, but the correlations be¬ 
tween the piimary parameters aie altered The inteipretation of the pn- 
mary factois lemains the same in both gioups 

In total selection, when the rank of the leduced conelation matrix is le- 
duced by 1, the simple structure may be distorted, depending on the lela- 
tion of the total selection to the primal y factors The mterpi etation of the 
pnrnaiy factois affected by the total selection may then be obscured 

When the new group is selected so as to satisfy conditions on two inde¬ 
pendent selection vanables, the variances of the selection variables are af¬ 
fected and also their covanances Such a situation is called multivariate se¬ 
lection Multivunate selection can be desenbed m teims of successive uni¬ 
variate selections The expected correlations after multivariate selection 
can be computed by matnx formulae dexived by Aitken The new correla¬ 
tions can also be determined by the univariate formula of Godfrey Thomson 
applied to represent successive univariate selections 
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With partial multivariate selection of a new group of subjects, the rank of 
the reduced coirelation matrix is augmented by one or more incidental fac¬ 
tors The number of incidental selection factors is determined by the num¬ 
ber of independent variables that participate in the multivariate selection 
The simple structure remains invariant, so that the primary factors can be 
identified with the same interpretation as foi the initial group of subjects. 
The additional factois may show appreciable variance, but they will not be 
mvanant foi differently selected gioups, since they are determined by the 
conditions of selection of each group If the attempt is made to interpret the 
incidental selection factors as basic parameters, then the interpretation will 
fail to be sustained m subsequent factorial studies of the same domain with 
differently selected subjects The primary factors should be identified m 
the differently selected groups 

With total multivariate selection the rank of the correlation matrix is re¬ 
duced by one or more factors, which depend on the number of successive 
variables on which the selection is complete, but the rank is also augmented 
by incidental selection factors, which depend on the number of vaiiables 
that participate in the selection With total selection involving the primary 
factors, the simple structure can be so distoited that the primaries may not 
be identified. For this reason it is well to allow as much variation as possible 
among the subjects m the domain to be investigated, thus practically elimi¬ 
nating the possibility of total selection within the parameters that aie to be 
sought In factorial investigation it is not of any consequence whether 
any of the groups of experimental subjects are representative of a general 
population The important consideration is that the experimental subjects 
should vary among themselves as much as possible within the domain that 
is being investigated 

The analysis of these various cases of selection is very encouraging, in 
that a simple structure has been shown to be mvanant under widely differ¬ 
ent selective conditions Hence the scientific interpretation of the primary 
factors as meaningful parameters can be expected to transcend the widely 
different selective conditions of the objects that are measured and factonally 
analyzed. This encouraging finding leads to a recommendation for the fac¬ 
torial study of any domain When a simple structure has been found for 
a test battery that has been given to an experimental population and when 
a plausible interpretation of the primary factors has been found, these 
should be regarded as hypotheses to be verified by giving the same test bat¬ 
tery to new experimental populations that should be selected in different 
ways If the primary factors are in the nature of basic parameters that are 
not merely reflections of the experimental conditions or the particular se¬ 
lective conditions, then these factors should be invariant under widely dif¬ 
ferent selective conditions, and their interpretation should be the same for 
the several experimental groups New test batteries should be constructed, 
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with prediction as to factorial composition of the new tests, and these should 
be tried on dfleiently selected subjects in order to determine whether the 
interpretation of each facto as a meaningful parameter can be sustained 
This is another situation in which the factorial methods depart from the 
conventions of statistical analysis It is customary in statistical reasoning 
to think of a general population fiom which we merely diaw samples, and 
the worry is then whether the sample is really representative of the uni¬ 
verse In facte analysis the principal concern is to discover an underlying 
order to be described in terms of meaningful parameters which should repie- 
sent scientific concepts The validity of a pnmary factor is determined by 
lb fruitfulness as a scientific concept It is inadequate if it serves merely as 
a regression coefficient When facte analysis has completed its job of 
charting a new domain, then it may be of practical importance to establish 
notms of performance for any specified general population Then the con¬ 
ventional statistical reasoning is again applicable, 


CHAPTER XX 
THE PRINCIPAL AXES 


Indirect solution 

When multiple-factor analysis was first developed, it consisted essen¬ 
tially in a shift of emphasis The dominant question in previous factorial 
studies had been as to whether the correlations could be accounted for by a 
single general factor and whether the general-factor residuals were large 
enough to justify the recognition of “disturbers,” which were called “group 
factors ” These weie sometimes frankly admitted at the start of a study, 
but always with the reservation that they should be regarded as secondary 
to the general factor, which was invariably postulated With multiple- 
factor analysis the fundamental question was formulated without this re¬ 
striction, namely, to determine how many factors were indicated by the ex¬ 
perimentally given correlations When that question was answered, the 
next question was naturally to inquire about the nature of all the common 
factors that determined the correlations. This extension of the factor prob¬ 
lem to any number of dimensions made it a question of fact as to whether 
one or more of the common factors were general in the sense of participating 
m the variance of every test in the battery. 

One of the early objections, raised by Spearman, against multiple-factor 
analysis was the indeterminacy of theproblem whenmorethanone factor was 
indicated by the correlations In the author's initial papers describing the mul¬ 
tiple-factor approach, this indeterminacy was shown to be associated with 
the question of where to locate the reference frame in the test configuration * 
The number of dimensions of that configuration was shown to be equal to 
the rank of the table of cori elation coefficients, which was defined as a square 
symmetric matrix f 

In the author’s first papers on multiple-factor analysis the indeterminacy 
of the location of the reference frame was resolved by placing the first refer¬ 
ence axis so as to maximize the sum of the squares of its factor loadings.} The 
next factor axis was located in the same manner by the first-factor residuals, 
and so on until the r principal-factor axes had been determined Each suc- 

• L L Thurstone, “Multiple Factor Analysis,” Psychological Review, XXXVIII 
(1931), 406-27 

fL L Thurstone, Theory of Multiple Factors (Ann Arbor, Mich Edwards Bros., 
1932), p 20 

t Ibid,., pp 17-19, 22-27. 
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cesswe reference axis was located so as to maximize the sum of the squares of the 
factor loadings or test projections This resolution of the indeterminacy of 
the reference frame was called the principal-axes solution * It has an impor¬ 
tant computational advantage, in that it is unique for any given correlation 
matrix This solution was written befoie the simple-structure principle was 
developed The principal-axes solution has considerable theoretical interest 
for factor analysis, and it is of practical importance m some types of factor 
problems 

The pnncipal-axes solution will be described here for three dimensions, 
but this solution is applicable to factor problems in any number of dimen¬ 
sions It will be assumed, fiist, that, the given correlation matrix R, has been 
factored into the factor matrix F 0 with elements a jm and that the reference 
frame of F 0 is arbitrary as determined by the factoring method that hap¬ 
pened to be used It may be assumed also that the diagonal elements were 
not adjusted for each factor. In this case the correlation matux with expen- 
mentally determined coefficients represents, in general, as many factors as 
theie aie tests, so that the nth factor lesiduals are identically zero when R is 
of ordei nXn The pioblem now is to rotate the arbitraly orthogonal refer¬ 
ence frame into that of the pi incipal axes Later the same solution will be dis¬ 
cussed as obtained dnectly from the correlation matrix without the arbi¬ 
trary intermediate factor matux, F a 

Let the unit vectors in each principal axis be denoted A P and let the direc¬ 
tion cosines of each of these unit vectors be denoted X„,„ The first 01 major 
principal axis is then identified by the unit vector Ai, where p = 1 with direc¬ 
tion cosines A ml and where the subscript m refers to the arbitrary reference 
frame of Fo The test vector j is defined by the direction numbers a ]m 

The projection of the test vector j on the major principal axis A p then 
is 

(1) V jAp = djlAip + 0,2X23, + 0,3X33, . 


The projection of 3 on the unit vector A„ can be denoted r, Ap because this 
projection is the correlation between test j and what may be regarded as a 
test A p of perfect reliability. 

Squaring the test projection on the principal axis, A P , which is to be de¬ 
termined, and expanding the terms for rank 3, we have 


( 2 ) 


j r,Ap a, 1 A l ,) -{- 0,2 X|p -f- cij 3X^2 -j- 2a,iO, 2 XipX2p T ,0, 3Xi p X 

1 + 2 a ,20,3X2pX3p . 


* The solution of this problem and of the chainctenstic equation was suggested to the 
writer in 1931 by Professor Walter Bnrtky, of the Department of Astronomy at the 
University of Chicago 
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Summing for the n tests j, 


^ *1 r 'iAp — ^ip ^ ^ Q Ji T - X; p C1J2 + X| p oj3 

1 i j 1 

-f- 2X : p \” p (ij{(i,« -|- 2XipX Jp 2X2 ; ,X;ip N a ]2 qj3 

1 3 3 

F 01 convenience, let 

(4) ^L 2 a? = m 

J 

Then the partial derivatives with respect to the direction cosine Xip of 
Ap is 

(5) ~ ^Xip^^^a,! d- 2 X 2 ^ ^ UjiCijZ 4" 2X3p^^n,iaj3, 

p 1 1 3 

and the other partials are written m the same manner 

The desired values of \ mp are subject to the restriction that A p shall be 
unit vectors Hence we have the conditional equation 



( 6 ) 


t; = X=p + Xip + X|p - 1 = 0 . 


By Lagrange’s method of undertermined multipliers we have the following 
normal equations 


(7) 


du , . dt3 

5Xip aXip 


du 


ax 


2p 


+ P 


dv 


ax 


2p 


0 , 

o, 


du dv_ 

l aXsp ax 3p 


The (r+1) unknowns are here the r values, X,„ p and /3 These are to be de¬ 
termined by the r equations (7) and the conditional equation (6) 

The partial derivatives of n aie 
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Substituting (5) and ( 8 ) in (7), dividing each equation by 2, and rear¬ 
ranging teims, we obtain 

Xi p (? O’f l + Pj + X 2 p + 1+2 + Xsp^^ + 1+3 — 0 , 

(9) S + 1+2 + x «( 2 ^* + 0 j + Xjp^^+g+a = 0 , 

J \ J J J 

Xi P ^ , + 1+3 + Xn p + 2+3 + Xj ?) ^ g^ 3 + /3^ = 0 . 

This is a set of r homogeneous linear equations in the unknowns X lp , Xi P , 
\ 3p The determinant of the coefficients m equations (9) must vanish, in 
order that a non-trivial solution may exist * Setting the determinant equal 
to zero, we have the characteristic equation 

(2a]i 4- /3) 2a } 1 * 2,2 2 + 1+3 

(10) A = 2+i+ 2 (2 + 2 2 -f- 18 ) 2+ a + a = 0. 

2+ 1+3 2 + 2 + 3 (ZaJj + /3) 

It will be seen that the coefficients +,„ of this deteiminant can be written 
111 the form 

(11) IP I = Iffol , 

wheie F 0 is a factoi matrix The characteristic equation then becomes 

(12) \F'F 0 + /3/| = 0 

or 

(13) |P+15/|=0. 

By expanding the deteiminant, this equation can be wiitten in the expanded 
foim 

(14) / 8 r + Ci(3 r_1 + Cnj3^ + + c r = 0 , 

which is of degiee r F 01 the piesent case the expansion is of thiid degiee, 
namely, 

(16) /3 3 + +/3 2 + c 2/ 3 + c 3 = 0 


* A trivial solution foi (9) is to set all the unknowns equal to zero 
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The coefficients of (15) can be written by expanding the characteristic de¬ 
terminant of (10), or they can be computed from the principal minors of P 
Each coefficient c x in (14) is the sum of all the arrowed principal minors of 
P. All the r roots of (14) are real and negative, and they will be denoted 
dll d^l • 1 P r 

Each of the roots dp can be substituted, in turn, in (9). Each of the r 
values of dp gives a set of direction cosines for a puncipal axis When a 
root dp is substituted in (9), the solution gives the direction cosines of A p , 
which are \i p , Ai P , \s p 

The principal axes A p are orthogonal Their direction cosines may be ai- 
ranged to form a matrix of the transformation A from the given orthogonal 
co-ordinates of F a to those of the principal axes This transformation may 
be represented by the equation 

(16) F 0 A = F p , 

where F p is a factor matrix for the principal axes Each value —dp is the 
sum of the squares of the projections of the test vectors on a puncipal axis 
A p These aie also the sums of the squaies of the factoi loadings in the col¬ 
umns of F v 

Numerical example of indirect solution 

In Table 1 we have a small correlation matrix R and a factor matux F 0 
with an arbitraiy reference frame It is desued to find the factor matrix F v 
for the principal axes The first column of F p will then be such that the sum 
of the squares of its loadings will be a maximum We first find the matrix 
P = F' 0 Fq, which is of order 2X2, since the rank of F 0 is 2 The expanded form 
of the characteristic equation (14) can then be written, and, since r = 2 in 
this example, the characteristic equation is the quadratic 

(17) d 2 + c,|3 + c z = 0 

The coefficient Ci is the sum of the one-rowed principal minors of P These 
aie merely the diagonal teims of P, so that 

(18) ci = 1 87 + 1 06 - 2 93 


The coefficient c 2 is the sum of the two-rowed puncipal minors of P There 
is only one such principal minor in this example, namely, the determinant 
| P | itself, and hence 


(19) 


= \P\ = 0 2133 . 
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The numerical form of the characteristic equation then is 


( 20 ) 


/3 2 -h 2 9300 /3 + 0 2133 = 0 . 


The two roots of (20) are —2 855297 and —0 074704 Both roots are real 
and negative Wnting equation (9) in numerical foim, we obtain 


( 21 ) 


(1 87 - 2.855297) Xi p + 1 33 X 2p = 0 , 
1 33 Xi p + (1 00 - 2 855297) X 2p = 0 , 


Table 1 


Factor Matrix 

Fo Correlation Matrix R. Factor Matrix F p 



I 

II 


1 

2 

3 

4 


Ai 

As 

1 

7 

3 

1 

58 

53 

71 

67 

1 

741049 

- 175632 

2 

6 

6 

2 

53 

61 

70 

71 

2 

758925 

184480 

3 

8 

5 

3 

71 

70 

89 

86 

3 

940456 

- 074454 

4 

7 

0 

4 

67 

71 

86 

85 

4 

919630 

065425 









—= 

2 855297 

- 074703 


Matrix P=I r oFo 


Transformation Maliix K 



I 

2 


Ai 

As 

1 

1 87 

1 33 

I 

803625 

- 595271 

2 

1 33 

1 06 

II 

595271 

803525 


IP| =0 2133 


|A|«+1 


fiom which we have 


Duection cosines of major 
principal axis Ai, 

where the subscnpt p = 1 m X mp The majoi principal axis has the direction 
cosines Xu and X 21 It can be identified m the latent roots because the nu- 
mencal value of —(3 is the sum of the squaies of the factor loadings of a 
column of F p The principal factoi p, for which 2 a? ]P is a maximum, is called 
the major principal axis The principal axis for which the sum of the 
squares 2 af p is a minimum is called the mean principal axis This axis is 
also of considerable interest in some factor problems 

The direction cosines of the mean principal axis for the present numerical 


( 22 ) 


X u = 803525 
X 21 = 595271 
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example are computed m the same manner from equation (13), in which we 
substitute the second laigest root /S 2 We then have 


(23) 


j (1 87 - 0 074704) Xi 2 + 1 33 X 22 = 0 , 
11 33 X 12 + (1 06 - 0 074704) X 22 = 0 , 


fiom which we have 


(24) 


J Xi 2 5952711 Direction cosines of mean 
( X 22 = 803525 J principal axes A 2 , 


wheie the subscript p = 2 in X mj) 

Having found the direction cosines of the two principal axes of the test 
configuration, we write the transformation matrix, A, which is 


(25) 


Ai A 2 


Xu Xl 2 


(+ 803525) 

(- 595271) 

X 2 i x 22 


(+ 595271) 

(+ 803525) 


This is an orthogonal matrix by which the test configuration can be de¬ 
scribed in terms of the principal axes instead of the arbitrary reference 
frame I, II, in F 0 We now have 

(26) P 0 A = F v , 


which is shown numerically m Table 1 It will be seen that A is an orthogonal 
matrix Its determinant | A | = 1 The sum of the squares of the columns of A 
are normalized The geometrical interpretation is that the pnncipal axes 
vectors are unit reference vectors The cross products of the columns of A 
are zero, which has the geometrical interpretation that the principal axes 
are oithogonal, their scalar products being zero 

The geometrical interpretation of the present example is shown in Figure 
1 , m which the test configuration is first drawn for the arbitrary orthogonal 
axes I, II, of F 0 , When the direction cosines of the major principal axis A! 
have been found (Xu, X 2 i), they can be lepresented on the diagram, wheie 
Ai is shown as a unit vector. The sum of the squares of the test vector pro¬ 
jections here is a maximum, as seems reasonable from the diagram, since the 
major principal axis passes through the central part of the test group On 
the diagram one may verify by measurement that the test projections on 
the principal axes agree with the numerical values in F P The second prin¬ 
cipal axis A 2 is also shown m the diagram. 

The numerical values of Xi 2 and X 22 in (24) can be reversed in sign, since 
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the homogeneous linear equation (23) defines only the ratio of X I2 to X 22 
Their numerical values are determined by the restriction that XJ 2 +X| 2 =i. 
If the signs of X t2 and X 22 in (24) were reversed, then the second column of 
the transformation matiix A would also be leversed m sign In Figure 1 we 
would then have the second principal axis reversed m direction The deter¬ 
minant of the transformation matrix would be leversed in sign, so that 
[ A | = — 1 When this detei ininant is equal to — 1, we have an odd number 
of reversals of direction of leferenee axes When | A| = +1, there is an even 
number of reversals of axes, or no reveisal. If the refeience frame I, II is 
subjected to rigid rotation in Figure 1 so that I goes to Ai when II goes to 
Aa, then the determinant of the orthogonal transfoimation is +1 If there is 
an odd number of reversals, this determinant is equal to — 1 Such is the 
case if we reveise the second column in A so that the reference vectors be- 


M 



come Ai and —A 2 It can be seen in the figure that the frame I, II cannot 
move to Ai and —A 2 by a rigid rotation within the dimensionality of r— 2 
For most factor problems the differentiation between the cases in which 
f A| = -(-1 and |A| = —1 is of no practical consequence It is of some in¬ 
terest to know the relation between the ambiguity of signs in equations (23) 
and the geometrical interpretation of the problem 

A direct solution for the principal axes 

In pi evious sections of this chapter we have considered an indirect deter¬ 
mination of the principal axes, in that the correlation matrix was first fac¬ 
tored into a factor matrix F 0 with an arbitrary reference fiame which de¬ 
pended on the particular method of factoring that was used on the correla¬ 
tion matrix. That was the author’s solution when the principal axes were 
first described The principal-axes solution can be written more directly m 
terms of the correlation matrix and without an intermediate factor matrix 
Eo 

Consider the correlation matrix R,^ and the desired factor matrix F p of 
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TghU 2 In this table the correlation matrix has only four rows and four 
columns, and the factor matrix has two factor columns It is desired to lo¬ 
cate the first principal axis so as to maximize the sum of the squares of the 
test vector projections on that axis. This means that the sum of the squares 
of the factor loadings a,i in the first column of F v must be maximized or 
that the first-factor residuals must be minimized The first-factor residuals 
can be written 


(27) t jk UjiUfci ~ Pjk ) 

where p,* is the residual Squaring the residuals, we have 

(28) r] k - 2a, 1 a k ir jk + ojiaji = p^k ; 
and, summing for all correlation coefficients, we have 


(29) SI = 

ok 3 k « k l 

Foi convenience, let 

(30) X s s 

; * 


Then 

(31) 


dz 

da,i 


-4^ yyaui 

k 


k 


Sotting the partial derivatives equal to zero, we obtain 

(32) 2r,fcati = a,i2afi . 

In the rest of this chapter the notation /3 will refer to the latent roots with 
positive signs Let the sum 2ali = /3, which is the largest latent root Writ¬ 
ing (32) m matrix form, we obtain 

(33) Ra = a/3 , 

where R is the correlation matrix, a is a column vector, and /3 is a scalar, 
namely, the first latent root 

Equation (33) is not directly a computing foimula, but it is the equation 
on which Hotelling’s iterative factoring method is based * 

* Harold Hotelling, “Analysis of a Complex of Statistical Variables into Principal 
Components,” Journal of Educational Psychology, XXIV (September and October, 1933) 
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Hotelling’s iterative method of factoring 

This method of faclonng enables us to obtain the desired factor matrix F p 
for the principal axes dnectly from the correlation matrix without the inter¬ 
mediate arbitral y factoi matrix F a Hotelling’s iterative method of factor¬ 
ing starts with a tual column vector Wi, so that (33) becomes 

(34) Rui = Vi>, 

wheie the product Ui is also a column vector The next tual vectoi, ii 2 , is 
taken proportional to Vi, so that u 2 = kvi Then 

(35) Rut = V 2 , 

which gives the new trial vector, u 3 =kv t , for the next trial The stretching 
factor 7c is arbitrary for each trial This process continues until the product 
numbers v Q are pioportional to the next preceding trial numbers u q to any 
required degree of accuracy The column vectors u„ and v„ are then both 
proportional to the desired column vector a Let the proportionality be 
lepiesented by 

(36) v, = pa , 
so that Ru=v becomes 

(37) R(pa) = (pap) 

The scalai product ( uv ) then is 

(38) (uv) — (pa) (pap) = p 2 /?2a 2 — p 2 /3 2 


Hence 

(39) 


v __ pap 
V (uv) V p 2 /? 2 


which ieduces v to the desired column vector a,i 

The latent root /3 can be determined at the solution by the simple rela¬ 
tion 


(40) 


P 


Sv 
2 u 


When the first-factor residuals have been determined by equation (27), 
the residual correlation matrix p,& can be factored by the same process to 
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give the second column of F PI whose elements are a j2 . The sum of their 
squares is 

(41) = 

3 

which is the second latent root This is Hotelling’s iterative method of fac¬ 
tor mg the con elation matrix It 1 is an improvement on the wi iter’s original 
method of finding the principal-axes solution by the indirect method of first 
finding an arbitrary factor matrix Fa and then rotating the reference frame 
to the principal axes The principal-axes solution can be written to repie- 
sent the whole factor matrix F p instead of one column at a time, as was done 
here Then the interpretation of the characteristic equation and the latent 
roots aie the same as in the writer’s original paper on the principal axes. 
Hotelling has renamed the principal-axes solution the principal components. 
Sometimes this term lefers to the factor loadings a tp in F p and sometimes 
to Hotelling’s iterative method of factoring This method and the writer’s 
original indirect method give numerically identical results, as will be seen 
in the accompanying example In computational work the indirect solution 
requires less laboi with the computing methods now available, but the di¬ 
rect solution is theoietieally the moie mteiesting 

Hotelling lestricts himself to the case of unit diagonals m the con elation 
matnx, but his method is applicable when other values, such as ichabihties 
or coinmunalities, are wntten into the diagonals It should be carefully noted 
that the lank of the correlation matrix, the number of common factors, and 
the latent roots are maikedly affected by the diagonal values The rank of R 
is lowest when communalities are used in the diagonal cells When unit 
diagonals aie wiitten, the rank generally becomes equal to the order of the 
correlation matrix 

An important characteristic of Hotelling’s iterative method of factoring 
is that the convergence is slow when the larger latent roots are nearly the 
same The convergence is rapid when the latent roots differ markedly in 
numencal values In the numerical example of this chapter we have chosen 
a case in which the latent roots are numerically quite different, so that the 
convergence is rapid and the solution is reached in a few trials The reader 
should not expect such rapid convergence m many practical problems m 
which the loots may have nearly equal numerical values 

Numerical example of Hotelling’s iterative method 

In Table 2 we have a small corielation matnx of oidei 4X4, whose latent 
mots are widely separated so as to give a rapidly convergent example of 
the iterative piocess The Gist tnal vector, Ui, is taken proportional to 
the column sums of R, so that the largest numerical value of «i is unity 
The matrix multiplication, Rui = v u is shown in the next column. The 
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recipiocal of the laigest entry m column v L is lecoided as the arbitrary 
stretching factor k, and we then have u 2 = fciq m the next column The ma¬ 
trix multiplication, Ru 2 =V 2 , is lecordcd in the next column. The proc¬ 
ess terminates when the two successive values u 2 and it a are found to be iden¬ 
tical within the required degree of accui acy 

The reduction of Vi to the desired column vector a, i is shown under the 
table The scalar product (u 2 u 2 ) is computed and recorded as shown The 
square root of this number is p/J, as shown Finally, the reciprocal l/p/3 is 
computed and recorded The final column a,, is computed from r 2 by the 
relation a^ — v/pfi 

The first-factor residuals aie then computed and are shown in the lower 
section of Table 8 The same process of factoring is applied to this table, 
with the resulting column vector a l2 as shown Finally, the two column 
vectors a ]p ai e recorded in the desired factor matrix F p The first ti ml vectoi 
for a lesidual correlation matnx can be taken proportional to the absolute 
sum of the residuals m each column, including or excluding the diagonals 
One can also justify taking as trial values the square roots of the diagonal 
residuals 

The factor matrix for the principal axes has two pioperties of interest 
The sum of the squares of each column of the factor matrix F p is a latent 
root Further, the cross product of each pair of columns is zero We then 
have foi the sum of the squares of each column 

(42) ^ a] P = 0 P , 

3 

and for the cioss pioduct of any pair of columns, p and q, 

(43) “ Q 

3 

Both these propeities can be easily verified numeiically in the factoi ma¬ 
trix F p 

The present example does not show the problem that arises when the con¬ 
vergence is slow The wide separation of the latent roots in this example 
produces the rapid convergence shown here When the latent roots are 
nearly alike, the slow conveigence makes it desirable to introduce other de¬ 
vices to increase the convergence to the desired solution 

The squaring process 

When the iterative method of factoi mg for the principal-axes solution is 
used on a coi relation matrix m which the largest latent roots are of neaily 
the same numerical values, the process converges slowly The labor required 
to reach the solution is then so considerable as to be discouraging The con- 
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vergence can be increased by the squaring process described by Hotelling as 
an improvement on the first form of his iterative factoring method * 

Consider the first iteration Ru 1 =v 1 (equation [34]), with a column vector 
ui which is either arbitrarily chosen as a start or chosen by inspection of the 
con elation matrix R If, now, this product iq is used as the next trial vector, 
we have 

(44) Rvi = v. 

Substituting (34) m (44), we get 

(45) R(Rui) = v, 
or 

(46) R 2 ui = Vi 

Using Vi as the next trial vector, we get 

(47) RVi = v a , 

and then 

(48) Rv 3 = Vi 
By (47) this becomes 

(49) R(Rv 2 ) = v t , 

and, by (46), 

(50) R l Ui = Vi 

This process can be continued, so that 

(51) R x u = v x , 

wheie x denotes any power of H If R is taised to a sufficiently high powei, 
the product vector v x is proportional to the desned solution a, i, even if the 
arbitrary trial vector is badly chosen A reservation may be made about the 
choice of the trial vector u. If this trial vector is exactly pioportional to any 
column of F v , including the secondary roots, the equation (51) is satisfied 
This situation is not likely to happen in practice, so that the iterations will 
piactically always lead to the major pi mcipal axis A simple example of this 

* Harold Hotelling, "Simplified Calculation of Principal Components,” Psyckometnka, 
I (1936), 27-35 
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effect m which iteration does not lead to the major principal axis is the fol¬ 
lowing 2X2 con elation matrix. 



Iteration with the tual vector (+1, 4~1), as shown, leads to the mean prin¬ 
cipal axis, whereas iteiation with the trial vector (+1, —1) leads to the 
major principal axis Examples of this kind arise very larely with experi¬ 
mental data, because it is very unlikely that a trial vector will be propor¬ 
tional to one of the secondaiy columns m F v 

From (46) it is evident that one iteration on W gives the same result as 
two iterations on R From (50) we see that one iteration on R i gives the 
same result as four iterations on R, and so on for any power of R. The more 
rapid convergence of this process is offset to some extent by the labor of 
multiplying the correlation matux by itself several times, and it must also 
be considered in relation to the fact that a large number of decimals are car¬ 
ried in this process m order to retain its full effectiveness 

The relation of this process to the characteristic root /3 can he seen if we 
write Ra — a/3 (equation [33]), where a is the desired column vector and /3 is 
the largest latent root of R or of a residual matrix This can be written 

(52) ^ Ra = a , 
from which we have 

(53) i rQ Ra) - a, 
so that 

(54) 1 R 2 a = a 

Repeating this process gives 

(55) R I a = (3 x a , 

from which it is seen that if R is raised to the power x, the iteration with a 
gives the product numbers (3 x a 

An interesting property of the correlation matrix is that when it is raised 
to a sufficiently high power it approaches unit rank to any required degree 
of approximation. * The factor matrix corresponding to such an augmented 

* Cyril Burt, "The Unit Hierarchy and Its Properties,” Psychometnha, III (1938), 
151-68 
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correlation matrix has only one column This symmetric matrix Ii x , which 
is of unit rank, can be factored by a single-column factor matrix, which may 
be represented by the column vector F X ~A, so that 

(56) R x = AA' ; 

and it is of some interest to see the relation between the single-column factor 
matrix A of R x and the first-column a of the factor matrix F p Let 

(57) a = pA , 

and substitute in (55), which becomes 

(58) R x (pA) = 0 *(pA) , 

and this can be written 

(59) pR x A = p() x A ; 

but R x can be factored by analogy with (55), 

(60) R*A = kA , 

where k is the laigest characteristic root of R x , namely, 

(61) k = A'A . 

Fiona (59) and (60) we get 

(62) pkA. ~ p(3 x A , 

and hence 

(63) k = P* 

We can then re-write (60) in the form 

(64) R*A = (3* A 

Fiom (56) we find 

(65) AA'A = p x A , 

and by (57), 

(66) i aa'a = - & x a , 

V 3 P 



and, since /3 = a'a, 
(67) 
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so that 

( 68 ) 

Hence by (57), 
(69) 


« Of d i 

p 6 p 


x-l 

fi 2 


l 

V 


1—x 1—x 

a = p 2 A = k 2x A, 


so that the desired column vector a n of R can be obtained from the single¬ 
factor loadings of R x This is not the usual procedure in the squaring proc¬ 
ess The generally recommended procedure is to continue the squaring until 
the column sums of two successive squarings, R x to R 2x , are proportional to 
any required degree of approximation The column sums of R x , or numbers 
proportional to them, are then used as a trial vector u on the original corre¬ 
lation matrix R, so that Ru = v The column vectors u and v may then be 
expected to be proportional, so that u or v is proportional to the desired 
column vector a,i If proportionality is not attained, the iteration may be 
continued on R The actual reduction from v to a is done by noting that the 
ratio 

(7°) J = P , 

u, 


and if one of the trial numbers u, is unity, then j3 can be read directly fiom 
the corresponding value of a. If one of the trial numbers u, is not unity, the 
root /3 can be determined from the ratio /3 = S | v | /21 u | for the last iteration 
A useful device, especially when it is desired to reduce a to a unit vector in 
the configuration before reaching the final principal-axes solution, is to com¬ 
pute the desired loadings from the relation 


(71) 


a, i = cv, 



where 



When the con elation matrix has been raised to some power x, so that the 
column sums of R x are proportional to the column sums of the next previous 
matrix R x/2 to the required degree of approximation, then R x is of unit rank 
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to the same approximation The single-factor loadings a, of R x can be found 
by the centroid method, which gives the unique values 


(73) 

in which 


a, 


__ m 3 t, 

~ vs; ’ 


(, = sum of absolute values of cell entries m row j of B 1 , 
m.j— ±1, aecoiding to the sign of s,, 
s, — algebraic column sums of R x 

The desired factor loadings can be obtained from the column sums s, 
of R x without an additional iteration on R If the column sums S x of R z , or 
the single-factor loadings a, of R x , are used in the iteration Ru = v, then 
it is expected that u and v will be proportional and that these will also be 
proportional to a,i Therefore, we write 


(74) a = pu 

Let Si denote column sums of R Then 


(75) 


2s iii = T , 


where T is the column sum of the product numbers Ru = V The ratio of 
this column sum T to the column sum 2u is the largest cbaiacteristic root 
Hence by equation (70), 


(76) 


jr 

Xu 


= 13 ■ 


From (75) we have 
(77) 


2si« 

Xu 


= 0 . 


From (74) we have 


(78) 


2 a? = (3 = p 2 Xu- , 


so that 
(79) 

and hence 


2s iU 
Xu 


= p 2 2w 2 ; 


V = 


i/ 


2s in 
Xu Xu 2 ' 


( 80 ) 
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By (74) we then have 


where u is proportional to the column sums of R x and s a represents the corre¬ 
sponding column sums of R It is good computational practice to verify the 
determination of a, 1 by the iteration Ra—^a for any method of obtaining the 
values of a An alternative procedure is to carry out the iteration Ru = v 
and then reduce v to by equation (71) Several relations have been de¬ 
scribed here, but it is not necessary in practice to verify all of them on each 
numerical problem m finding the solution a,i The same procedures are used 
on the correlation matrix residuals for determining each column of the factor 
matrix F v 

Numerical example of the squaring process 

In order to illustrate the squaring process, we shall use a small example 
which does not converge so rapidly as the previous example with Hotelling's 
original iterative method of factoring This example is shown in the factor 
matrix F p of Table 3. The two characteristic roots are 1.26 and 33 as shown 
The two columns represent principal axes, as can be seen from the fact that 
20,10,2 = 0, The coriesponding correlation matrix R is shown in the same 
table 

The expected column sums s 3 of IF are computed from R, and, since these 
are not proportional to the column sums of R, the complete matrix R 2 is 
computed. Under each matrix we have reduced the column sums s, to the 
proportional values ks 3 , in which the largest value is unity This facilitates 
comparison of successive sets of column sums 

The expected column sums of R 4 are computed from R 2 , and these are 
again not proportional to those of R 2 , so that R i is computed in full as shown 
Next we compute column sums of i2 s and compare with column sums of R' 
These two sets approach proportionality, but they still differ in the second 
decimal Hence R s is computed as shown The column sums of R 16 agree 
closely with those of R s , and hence it is not necessary to compute R u in 
full The discrepancies here do not exceed 0002 

In Table 4 we have a summary of numerical examples. The trial values 
u = ksj from R 16 are used in the iteration Ru = v, as shown The computation 
2uw = 3 2400000 leads to the values of a, i by equation (71) 

In the same table we have the computation of a, t from column sums of 
R 15 In the first column of this table we list u = ks le of R IC The values of a,, 
are then obtained from equation (81) Eor check purposes these values 
should be verified by the iteration Ra = f3a for one or more rows of R 
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Correlation. Matrix R 



I 

II 
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7 

0 

1 

49 
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35 
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2 
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— 4 

2 

- 42 

52 

- 14 

20 

3 

5 

- 4 

3 

35 

- 14 

41 

- 24 

4 

- 4 

1 

4 

- 28 

20 

- 24 

17 

2a 

2 

- 7 

s t 

14 

16 

38 

- 15 

2a 2 

1 26 

33 
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37 

42 

1 00 

- 39 


Matrix R® 
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3 
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- 5292 

4410 

— 3528 

2 
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- 3252 
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3 
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- 3252 
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- 2652 

4 

- 3528 
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2049 

s, 

1764 

- 0588 

2184 

- 1239 

ks, 

8077 

- 2692 

1 0000 

- 5673 


Matrix R' 
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3 

4 

1 

98018424 

- 84015792 
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— 56010528 

2 

- 84015792 
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- 59436288 

47865276 

3 
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- 59436288 

50584392 

— 40151268 

4 

- 56010528 

47865276 

- 40151268 

32041953 

S 7 

28005204 

- 22998276 

21009996 

- 16254567 

ksj 

1 00000000 

- 82121261 

75021596 

- 58041113 


Matrix R s 
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2 


3 


4 

1 

2 

47052866 

-2 

11759599 

1 

76466333 

— 1 

41173066 

2 

-2 

11759599 

1 

81515047 - 

1 

51250038 

1 

21003780 

3 

1 

76466333 

-1 

51250038 

1 

26054199 

— 1 

00839609 

4 

-1 

41173066 

1 

21003780 - 

1 

00839609 


80670750 

Sj 


70586534 

_ 

60490810 


50430885 

— 

40338145 

ks. 

1 

00000000 

— 

85697380 


71445476 

— 

57147083 
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7 

76451864 
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65528464 

5 

54610179 

4 

43687205 

Matrix R 10 

s, 

4 

48421281 

-3 

84361097 

3 

20300917 

—2 

56240732 

ks } 

1 

00000000 


85714285 


71428572 

— 

57142857 
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The principal-axes residuals 

When one factor column of F p has been determined, the extraction of the 
next factor from the correlation matrix is usually done by computing the 
lesidual correlation matrix, 


(82) 


Ri = R — act' , 


where Ri denotes here the first-factor lesidual matrix from R These lesid- 
uals can be computed in the same manner as for any other factoring method 
The residual correlation matrix is then used for determining the next factor 


Table 4 

Computation of aji by Iteration Ru = V 



U 

13 

a#i 

1 

1 00000000 

1 26000000 

70000000 

2 

- 85714285 

-1 08000000 

- 60000000 

3 

7142S572 

90000000 

50000000 

4 

- 57142857 

- 72000000 

- 40000000 


Suv = 3 24000000 
1/Xuv= 30864198 
1/Vzuu = 5555S556 


Computation of a« from Column 
Sums u = ksu of R ia 



U 

0,1 



1 

1 oooooooo 

70000000 

Xu = 

28571430 

2 

- 85714285 

- 60000000 

Xu 1 =2 

57142857 

3 

71428572 

50000000 

XSiU~ 

36000000 

4 

- 57142857 

- 40000000 

V 2 - 

V = 

48999998 

70000000 


column a l2 of F v The procedures on R\ are the same as those which have 
been described for the first factor 

In addition to the customary method of computing residuals, we shall de¬ 
scribe two methods, namely, (1) Hotelling’s adaptation of the squaring 
process to the computation of residuals and (2) Tucker’s method of comput¬ 
ing the subsequent principal-factor loadings without computing residuals 
If the squaring process has been used in the determination of the first- 
factor loadings a, i of F v , then the augmented correlation matrix R x can be 
used also for writing the residual correlation matrix raised to the same power 
with considerable reduction m computational labor For this purpose we 
shall first prove several useful relations 
Let 


(83) 


Q == ao' 
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where Q is a square symmetric matrix of unit rank produced by the column 
vector a, which has been determined The residual matrix R ± is then 

(84) Ri = R - Q = R - aa' 

We also have the two relations previously established, namely, 

(33) Ra = /3 a , 

and 

(85) 0 = a'a 
From (33) and (83) we have 

(86) RQ = Raa' = 0aa' , 

(87) RQ = 0Q . 

From (83) and (85) we can write 

Q l = ( aa'){aa') 

= a{a'a)a' 

= a/3u' 

= p aa' 

(88) Q 2 ~ pQ . 

Applying these relations to the augmented correlation matrix R*, we 
have, starting with R 5 and using (87), 

R 3 Q = R 1 (RQ) = R 2 pQ = 0R*Q 
= pR(RQ) = 0/2(/3Q) = 

= 3 2 (PQ) 

(89) = P 3 Q , 


and, in general, 
(90) 


R*Q = 0 X Q 
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Another relation involving R x can be shown from (87) and (88) by writ¬ 
ing 


R 3 Q = R 2 (RQ) = R 3 Q 3 
= R(RQ)Q = RQ 2 Q 

(91) R 3 Q = RQ 3 , 
or, in general, 

(92) R X Q = RQ* 

Powers of Q can be reduced from (88) by writing 

Q 3 = QQ2 = QtfQ) _ PQ2 , 

(93) Q 3 = /3(/3Q) = /3 2 Q , 
or, in general, 

(94) <2* = /S^Q 

The residuals can be expressed by (84), (87), and (88) from the squarmg 
process by writing 

R\ = (R - Q) 3 = R 2 — 2RQ + Q 2 
= H 2 — 2/3Q + /SQ , 

(95) Rl = R 3 - PQ, 
or, in general, 

(96) R? = E* - /S^Q , 

by which the residuals Rf raised to the power x can be obtained from the al¬ 
ready computed matrix R\ In using (96) the procedure is as follows. 

1) The col umn sums s of R x are used as trial numbers in the iteration 
Rs = v Several iterations are taken, if necessary, until a n are de¬ 
termined 

2) Determine characteristic root p 

3) Compute the product numbers R x o = /3 x a, in which 
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4) Determine /3 r 1 by the division 



5) Compute R\ by equation (96). 

The squaring process has advantages in rapid convergence, but it is also 
quite laborious in practice, especially when large correlation matrices are 
to be raised to a high power 

Another solution for the principal-axes problem without residuals has re¬ 
cently been proposed by Ledyard Tucker * It is ingenious and looks prom¬ 
ising, but it has not yet been tried extensively In Table 5 we have a 4X4 
correlation matrix Ri, which gives the first principal-factor loadings m the 
adjacent column a, t i The sum of the squares of this column is /3i= 81 
When this first factor has been determined, the usual procedure is to write 
another matrix of correlation residuals The present method avoids the 
necessity of computing residuals Instead, the original correlation matrix is 
merely bordered, as shown in Literal notation at the top of Table 5 The in¬ 
dicated elements of the bordering column and row have the effect of reduc¬ 
ing the largest characteristic root to zero. Then the major principal factor 
of the bordered matrix 72 2 is the second principal factor of the original ma¬ 
trix 72 1 The corresponding factor matrix F p is also shown m literal form at 
the top of Table 5 with an added row The sum of the squares of the first 
column of this factor matrix is identically zero, so that the major principal 
factor now becomes the second column of F p 

The bordered matrix 72 2 is shown in numerical form in the same table 
Hotelling’s iterative process on this correlation matrix gives the second- 
factor column 0 , 2 , as shown, with the second characteristic root ^ 2 = 49 
The matrix 72 3 is constiucted by bordering 72i with two additional rows and 
columns Hotelling’s process on this matrix 72 3 gives the third-factor column 
o ,5 The three factor columns are assembled in the numerical factor matrix 
F P at the bottom of the table 

Graphical aids in factormg the correlation matrix 

In the iterative process of determining the principal axes the computa¬ 
tions can be reduced by graphical methods, at least in the first few trials | 
The iterative process depends on the relation 

(97) Ru = t;2w 2 , 

* “The Determination of Successive Principal Components without Computation of 
Tables of Residual Correlation Coefficients," Psychometnka, IX (September, 1944), 
149-53 

|LL Thurstone, “Graphical Method of Factormg the Correlation Matrix,” Pro¬ 
ceedings of the National Academy of Sciences, XXX, No 6 (June, 1944), 129-34 
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where R is the given, correlation matrix, u is the trial column vector, and v 
is here a column vector proportional to the product Ru Consider in particu¬ 
lar the scalar product of row j of R ]k and the trial column vector u. This 
scalar product is 

(98) RjU = v/S,u 2 , 

from which we have 


(99) 


v i 


R,u 

5 


but this is also the well-known equation for the slope of Lhe regression line 
r on it through the origin foi a plot of r against u If we plot columns of r jk 
against the trial values u k , we get a plot with n points The best-fitting 
straight line through the origin (regression r on u) can be diawn by inspec¬ 
tion, and the slope of the line is easily read graphically with sufficient ac¬ 
curacy for the first few iterations That slope is the value v, To plot, say, 
thirty points and to draw by inspection the best-fitting straight line through 
the origin takes less time than to obtain the cumulative sum of thirty cross 
products on a calculating machine 

By equation (97) it is evident that at the solution the values v, = u, — a ji 
When a set of values of v has been determined graphically, we plot v against 
u and find the slope m In the first few graphical iterations, this slope will 
not be unity because the trial values u are probably too large or too small 
If the true values are better represented by ku, m which k is a stretching 
factoi, then a plot of r against ku will give a slope of v/k instead of v When 
these graphs have been drawn, the slope of v/k against ku should be unity. 
But the obtained slope of v against u is m for the values actually used 
Hence the observed slope m = k 2 Having found the slope m of the plot of 
v against u, we find k = Vra Then if we should take a new set of trial num¬ 
bers Xj — kuj, the iteration would give, instead of equation (99), 

(ioo) fe = y, 


where y = v/k The slope of y against x should now be unity, and the values 
of y should be used as the trial vector for the next graphical iteration The 
determination of m and k can be done with a slide rule, and the new trial 
values y — v/k can be similarly determined The slope m can be found either 
by inspection or by simple summing with the method of averages, m which 
m = Xv/Xu for like signed pairs of v and u 

A numerical example is shown m Table 6, which starts with the correla¬ 
tion matrix R of order 10 X10 and rank 3 The corresponding factor matrix 
F p for the principal axes is shown in the same table It is the solution to be 
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found from B The latent roots of R are 2 85, 1 47, and 0 63 These are the 
sums of the squares of the columns of F P . In the same table the initial trial 
vector u for the major principal factor is taken roughly proportional to the 
sums of the columns of R. Column 1 of I? is plotted against column Ui, and 
the slope is estimated to be approximately -f- 05 and is recorded in column 
tii Ten such plots give the values m column a. The slope ib read directly 
from each graph by noting the ordinate of a straight-line fit at ui = 1 Next 
obtain the two column sums, Stii and The ratio 2i>/2-u = m, and 

k = Vm This summation can be absolute sums for like signed pairs of u 
and v. Then compute i/i —Wi/fc with a slide rule These are also the trial 
values u 2 for the next trial Proceed likewise for three trials, which give the 
desired values of a, l to two decimals 

The first-factor residuals aie then computed These are p 3 k — 'r, 1c ~a 1 ia k i. 
Choose as a starting vector that column of the first-factor residuals which 
has the largest absolute sum, ignoring diagonals This is column 1 Its val¬ 
ues are recorded m column Ui for factor II The procedure is now the same 
as before. The thud iteiation was taken on a calculating machine, and it 
gave the second-factor loadings Oj 2 , the second column of F r , to two deci¬ 
mals The last iterations for each factor can be done on a calculating ma¬ 
chine to obtain greater accuracy, while the first few iterations can be done 
graphically to save time and labor The computation for the third column 
of F p was done m the same manner 

Occasionally, when two or three latent roots are nearly the same, the iter¬ 
ative process will be found to oscillate, or the convergence will be slow This 
is an indication that the test configuration has neatly equal thicknesses in 
these dimensions, so that the configuration is nearly circular or sphencal m 
these dimensions If the purpose is to extract the maximum variance from 
the correlation matrix, it does not matter where we place a set of orthogonal 
axes in those dimensions which are represented by nearly equal latent roots 
In such a situation it is desirable to reduce the obtained vector a to a unit 
vector in the system without waiting for complete convergence That can 
be done with a stretching factor c on v so that co = a, where a are the desned 
factor loadings The value of c can be found fiom equation (72) By this 
device the computer need not be unduly delayed by slow convergence in 
obtaining a factorially useful solution The graphical method here, described 
can be applied to a variety of computational problems. 

Additional theorems 

We shall consider here some additional properties of the principal-axes 
solution which are of theoretical and computational interest * Starting 
with the relation of equation (32), we write 

( 101 ) RjkQ'ii ~ 

* The theorems m this section were assembled by Ledyard Tucker. 
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for the first column a ,i of the factor matrix F p for the principal axes For 
the second column we have, similarly, 

(102) R^a,! = a, 2 P 2 , 

and, m general, 

(103) Rjk&jp — (ijpfSp 

The elements of F p are a, p , and the characteristic roots ftp may be taken as 
diagonal elements m the diagonal matnx D Then, in matrix form, 

(104) RF P = FpD . 

The matrix F p is orthogonal by columns, so that a Jp a Jr , — 0, where p and q 

3 

represent any pair of columns Since aj p =ft p , the columns of F P can be 

3 

normalized by the postmultiplication The matrix F p D~i is then an 

orthogonal matrix, which may be denoted A, 

(105) F p D-i = A or F P = AD 1 , 

by which we see that the columns of A are proportional to the columns of the 
factor matrix F p From (104) and (105) we have 

(106) RFpD-t = F p DD-l , 
and hence 

(107) if A = AD . 

Premultiplymg (107) by A', we have 

(108) A'ifA = A'AD ; 
but, since A is an orthogonal matrix, we get 

(109) A'ifA = D 
or 

(110) if = ADA' 

Since AA' = I, we get from (105) 

(111) F'pFp = D . 

Several useful relations of the characteristic roots ft p are as follows The 
sum of the squares of each row j m any factor matrix F is the communality /i). 
Hence the sum of the squares m the whole factor matnx F is equal to the 
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sum of the diagonal elements in R The sum of the squares in each column p 
of F p is the root p P Hence the sum of the squares of all the elements m F p 
is equal to the sum of the roots /3 P) and this is also the sum of the diagonal 
elements of R 

The sum of the squares of the correlations m R is equal to the sum of the 
squares of the entries m the matrix P — F'F, where F is any factor matnx 
for R We write 

( 112 ) Tjk = (Lj m ClLm , 

m 

and then 

( 113 ) 22*= 22 

J k 3 k 

which becomes 

aw) 22*-22(2 

j k m M \ j 

or 

(ns) 22*= 22 

3 k m M 

but 

(116) P = F'F , 

m which the elements are 

(H7) VmM = 2 

1 

and hence 

( 11S ) 22* = 22^' 

1 h m M 

For the principal axes we have, by (105), 

(119) F' p Fp = (AZM)'(AD») 

= £> 5 A'AD 4 . 



But A is an orthogonal transformation, and hence 
(120) F' V F V = D = P v 
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Hence the diagonal elements of P are the characteristic roots of R. It can 
be shown that the char aeteratie roots of R and P are identical 
If, in applying Hotelling's iterative method of factoring, the first trial 
vector is orthogonal to the first s principal axes, it can be shown that all suc¬ 
ceeding timl veotoiri will also remain orthogonal to the first s principal axes 
and they Will appmarh the (x-f-ljth principal axis 

It can also be shown that if we subtract a number k from each diagonal 
element of R, then the elwu uetei ist.ie roots are reduced by the same scalar k, 
but the, solution A of equation (111)) remains unaltered. The introduction 
of the decrement. Ic on the diagonal elements of R can be used for accelerat¬ 
ing the iterative pineess. 'Hie iterations ean be made to appioach the mean 
principal axis ms tend of (lie major piinripal axis by this device. The mean 
principal axis ean also be determined by iteration on P~ l , which has the 
chaiac’teiistie mots 1 d,„ and these aie the diagonal elements of D~ l 

Advantages and limitations of the principal-axes solution 

The pi mi lpal-axes solution for the factor pioblem has an mtiinsic inter¬ 
est because of its many mat licmutical properties, including the fact that it 
is a unique solution foi any given eunelation matrix That was the writer’s 
motivation m t.lteongmal foimulation of the principal-axes solution in 1932 
Soon aflei publication of (lie .solution, it was leahzed that the solution did 
not satisfy the psychological loquuemonts for which the simplc-structure 
solution was devised Hoeutiso of I lie i at,her general confusion about the con¬ 
troversial issues in 1 ins pinhlcm, we shall summarize briefly the advantages 
and the limitation* of the pi ineipal-axes solution, including a description of 
the types of problem foi which the principal-axes solution is applicable and 
of those to which the pmw ipnl-nxes solution is not applicable. 

A factorial analysis ean be made for one of two purposes, namely, (1) to 
condense the test si mo by expicasing them in terms of a relatively small 
number of lmeiulv independent factors or (2) to discover the underlying 
functional unities w Inch opetate to produce the test performances and to 
describe the individual diftci cnees eventually in terms of these distinguish¬ 
able functions. Many mgnnients about factorial methods could be resolved 
if the part icipants would indicate which purpose they have in mind, because, 
strictly speaking, tliei e is no intrinsically coirect factorial method except m 
tonnsof what, the investigator is tiyingto do According to the first purpose, 
factor analysis becomes essentially a statistical method of condensing the 
lest seoies. Aceorihng to l hi* second pm pose, factor analysis becomes essen¬ 
tially a scientific method for sustaining or rejecting hypotheses about the 
nature of (he amici lying pineesnes that produce the test scores These two 
points of view are hot h legitimate in their respective places. There is trouble, 
however, when two investigatois have these different points of view and 
proceed to aigue about, factorial method without recognizing that they rm- 
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ply entirely different objectives with their analyses It is the second objec¬ 
tive that has guided the writer, since 1931, in developing multiple-factor 
methods It is the first objective that is usually implied by mathematical 
statisticians who wnte on factorial problems This difference m objectives 
is largely responsible for controversies about factorial method Many mis¬ 
understandings m this field would probably be resolved if we all recognized 
that a factorial analysis should be judged in terms of the purpose for which 
it is made and that multiple-factor analysis is a flexible and powerful tool 
that can serve as a guide in a variety of scientific problems 

Let us consider, first, a table of scores s 3l for individuals i m tests j This 
is the score matrix of order nXN For convenience let the scores be recorded 
m standard foim, with unit standard deviation and with origin at the mean 
This table shows n scoies for each of the N individuals If the tests are cor¬ 
related, we may undertake a factor analysis m older to condense a large 
score matrix so that each individual may be descubed as to the same domain 
in teims of a relatively small number (r) of linearly independent factor 
scores instead of the large number (n) of test scores with which we start m 
the score matrix This is essentially a statistical problem, and, as such, there 
is no demand that the factor scoies shall have any particular interpretation 
in terms of the processes that produce the scores This situation is very 
similar to that of writing a 1 egression equation m which we want a set of co¬ 
efficients for the independent variables which will give that linear combina¬ 
tion of them which minimizes the residuals in estimating the dependent 
variable When we solve the least-squai es problem and write a regression 
equation, we demand only that the regression coefficients shall be numbers 
which minimize the residuals We usually make no demand that the regres¬ 
sion coefficients shall be interpreted in terms of the processes which pro¬ 
duced the dependent variable scores If that is evei done, we can expect 
each of the 1 egression coefficients to be complex in nature, m that it is af¬ 
fected by all or by many of the processes which affect the problem How¬ 
ever, it is just as legitimate to ask for a condensation of the test scores into a 
relatively small number of indices to replace the original test scores as it is 
to ask foi a linear combination of the independent variables for the purpose 
of making the best-possible prediction of the dependent variable, without 
any reference to the question of what the regression coefficients might mean 
in terms of the processes that fundamentally determine them 

Starting with the scoic matiix of otdor nXN, it is our object, according to 
the first purpose of factor analysis, to write another scoie matrix of order 
rXN in which r<n, and we want to describe each individual in terms of this 
smallei numbei (r) of indices so as to cover essentially the same domain 
which is coveied by the larger numbei (») of tests in the whole test battery 
The fundamental factor theorem which was first formulated m 1931 applies 
here, namely, that the number of linearly independent factors in a correla- 
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tion matrix is the rank of the correlation matrix.* The correlation matrix 
for this statistical problem has unit diagonals because the correlation ma¬ 
trix R — (1/N)SS', where the elements of S are standard scores 

The score elements m S are fallible, m that they are experimentally de¬ 
termined, and hence there is at least some part of the variance of each test 
which is fortuitous and attributable to variable errors The rank of the cor¬ 
relation matrix with experimentally determined correlation coefficients is 
necessanly of rank equal to the order of the correlation matrix Hence, 
strictly speaking, n = r with experimentally determined correlations. If we 
cany out the factor analysis with this literal interpretation, then we shall 
have as many factors as there are tests We might as well use the tests as 
they stand for Lhe description of each individual, since we have gained noth¬ 
ing m the economy of description However, there is a gam in the statistical 
sense m that the new factors would be uncoirelated, whereas the tests are 
correlated. The investigator may consider it worth while to use a set of n 
uncorrelated factor scores for each individual instead of a set of n coirelated 
test scores That may be his choice 

Economy of description of each individual can be obtained in most prac¬ 
tical problems, so that r is less than n, even with the present formulation, 
because it may be possible to select a relatively small number of factor 
scores or indices which cover most of the variances of the tests in the bat¬ 
tery This is, in fact, the objective when the principal-axes solution is used 
without rotation of axes The investigator then makes a factor analysis in 
terms of a relatively small number of factors which account for the test 
scores with residuals that are so small that he is willing to ignore them The 
number of factors to be used in such a description depends on the size of the 
residuals that are judged to be negligible 

In the wi iter’s first attempt to solve this problem, the first column of the 
factor matrix was so determined that it accounted for the maximum possible 
amount of the variance in one factor Geometrically, this solution is found 
by placing the first axis m the test configuration so as to maximize the sums 
of squares of the test vector projections on the first axis This is, in fact, the 
major principal axis The next column was so determined that it accounted 
for the maximum possible amount of the variance from the first-factor resid¬ 
uals The process can be continued until the residuals are negligible It is 
evident that the pnncipal-axes solution is exactly the desired solution for 
this problem At first, it was necessary to factor the correlation matrix with 
an arbitrary orthogonal reference frame, which was rotated to the principal 
axes. Then, only the first few principal axes were used to describe the test 
battery, leaving the remaining variance unaccounted for in the form of resid¬ 
uals With the publication of Hotelling’s iterative method of factoring, it 
*L L Thurstone, “Multiple Factor Analysis,” Psychological Review, Vol. XXXVIII, 
No. 6 (September, 1931) 
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became possible to determine the principal axes directly from the correla¬ 
tion matrix, one at a time and starting with the major principal axis The 
factoring need continue only until the residuals are j udged to be negligible 
The solution is in such a form that we can say with confidence that no other 
set of r linearly independent factors can be determined for the given set of 
test scores which accounts for moie of the variance In this statistical sense 
the first r principal factors constitute the best description that can be found 
for the Lest scores with this number of factors In practice, there is a limita¬ 
tion of excessive labor with a large eonelation matux A piactical compro¬ 
mise is to determine a factor matrix with one of the centroid methods and to 
determine one or two more factors than would be required by the principal 
axes in order to cover the same total variance This centioid frame can then 
be rotated to the principal axes when that is the desired solution 

We consider, next, the second purpose for which a factorial analysis can 
be undertaken, namely, to discover the underlying functional unities which 
operate to produce the test performances A verbalism is frequently given 
as an objection to the meie statement of this purpose, namely, that no meth¬ 
od of analyzing data can discover factors Let us meet such a triviality by 
acknowledging that no method of analyzing data can ever diseovei any¬ 
thing m science or give any interpolation of the data Discovering any¬ 
thing 01 interpietmg experimental findings of any kind always implies the 
subjective functions of a human being, the expoiimenler Analytical meth¬ 
ods can aid his thinking, but they never wi ite the interpretations for him 
The analytical methods that he selects depend on his purposes 

As an example of the second purpose, namely, to discover functional 
unities and their nature, let us consider a set of tests of visual perception 
The correlations between the tests might indicate that more than one factor 
is involved and hence that visual perceptual efficiency cannot be adequately 
described as a single function but rather as a complex of several or of many 
functions On the basis of previous factorial experiments, two visual func¬ 
tions, such as visual acuity and visual perceptual closure, might be postu¬ 
lated among many other factors Every visual test will mvolye visual acuity 
to some extent in its variance, but, if the figures to be seen are reasonably 
large and distinct and sufficiently illuminated, the factor of visual acuity 
will be represented in only a small and negligible part of the total variance 
m studying individual differences m such tasks. Similaily, the factor of 
visual perceptual closure will be represented in most visual tasks, but most 
of the tests may have little or almost none of then variances accounted for 
by individual differences in visual closure If such factois operate to produce 
individual differences in perfoimanee on visual tasks of considerable variety, 
it should be possible to predict with some confidence that the relative vari¬ 
ances will be large or small in these two factors by merely examining the 
psychological nature of each test If the battery of visual perceptual tests be 
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considered with special reference to these two postulated factois, apart from 
the other factors that are undoubtedly present, then we shall have a fan¬ 
shaped configuration of test vectors in the plane of the two postulated 
factors Toward one edge of the fan we should find those tests which have a 
large part of their variance attributable to one of the postulated factors, 
and toward the other edge of the fan we should find those tests which are rel¬ 
atively more saturated with the other postulated factor This is, of course, 
the simple-structure principle m two dimensions If it is found that the test 
vectors really do lie in such an arrangement m the test configuration and if 
their relative positions agree with the psychological interpretation of the na¬ 
ture of the tests, then we shall have experimental evidence that sustains the 
factorial interpretation, but here, as elsewhere m science, no hypothesis can 
ever be proved to be correct by any experiment It can only be shown to be 
plausible If, on the other hand, the test vectors do not oblige ns by falling 
into the predicted fanlike structure but fall, instead, into some other entirely 
different arrangement, then we have evidence that our postulated factors do 
- not agree with the functional unities that operate to produce the test per¬ 
formances We must then guess again as to the possible nature of the under¬ 
lying order This is, in fact, the rule in factonal experiments They usually 
tell us, in effect, that we must guess again. When a factonal hypothesis is 
sustained, it is a challenge to investigate further with the leads whose plausi¬ 
bility has been indicated factonally An example of such a failure by the 
writer was a hypothesis that the general intellective factor was a matura¬ 
tion factor Factorial results at different age levels have recently shown 
that such a hypothesis is probably wrong 

In setting up factorial experiments it is well to design the tests so that the 
factorial results may have an opportunity to give us crucial tests of a hy¬ 
pothesis or m the choice of rival hypotheses. One such pattern of great use¬ 
fulness is to design tests which clearly include the postulated factor and 
other tests in the same domain, as similar as possible, which do not involve 
the postulated factor m the test variance Presence or absence can then be 
predicted for the postulated factor in each test Those tests which do not 
involve the factor should lie at one edge of the fan in two dimensions, in one 
plane in three dimensions, or in a hyperplane m n dimensions Furthermore, 
all the tests which do involve the postulated factor m their variances should 
all lie on one side of the hyperplane and not on both sides of it This is 
again the simple-structure principle in analyzing the test configuration 
Scientific interest m a simple structure, when it is found, is relative to the 
state of knowledge at the time of the experiment For example, if we should 
set up some tests of visual perception and some tests of auditory perception, 
we should not be surprised to find that the visual tests had something in 
common which was entirely absent m the auditory tests We might then 
call the factors merely visual and auditory, and that would be a first rough 
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guess, which would, no doubt, be refined with further experiments That 
there should be distinguishable functional unities for these two modalities 
is probably taken for granted, so that the rough factorial experiment would 
be of little interest We should know such a result before we started On 
the other hand, the clean separation between the verbal factor V and the 
word-fluency factor W was not at all evident when several crucial tests were 
devised to test the hypothesis that at least two such factors were responsible 
for some of the individual differences in verbal tasks Two synonyms tests 
were devised, with the prediction that one would be high m V and zero m W 
and that the other synonyms test would behave m the opposite manner 
factonally The prediction was sustained, and this constituted a lead in the 
further investigation of verbal functions in human intelligence. 

Now suppose that the factonal analysis was made by determining the 
principal axes and that the solution was left in that foim without rotation 
of axes to simple structure. The test scores would, indeed, be accounted for 
by the smallest possible number of factors, and the residuals would then be 
minimized by the well-known method of least squares In the two-dimen¬ 
sional example the major principal axis would be in the middle of the fan of 
test vectors, and the second pnncipal axis would be arbitranly at right an¬ 
gles to the first axis Such a resolution does not give us the information we 
are looking for if our purpose is to investigate hypotheses concerning the 
underlying processes that determine the individual differences xn peiform- 
ance It should be evident that the principal-axes solution is the ideal solu¬ 
tion for the problem of statistical condensation of the test scores into the 
smallest possible number of factors but that this solution is inadequate for 
the analysis of the underlying processes and their identification 

We have considered the principal-axes solution with reference to two dif¬ 
ferent purposes for which a factor analysis may be undertaken. There are 
several situations in factor analysis in which the principal axes are of inter¬ 
est, apart from the major purposes for which the analysis is made Even 
though we discard the principal axes as the final factonal resolution to be 
studied for scientific interpretation, the principal axes are, nevertheless, of 
fundamental interest Consider the problem of analyzing a correlation ma¬ 
trix with experimentally determined coefficients Because of the fact that 
every coefficient in the table is determined m part by fortuitous variable 
errors, we must recognize that the total variance cannot be accounted for in 
teims of a limited number of factors whose interpretation may constitute 
the major pui pose of the analysis A part of the variance must necessarily 
be assigned to the variable errors The first task is to factor the correlation 
matrix in terms of a limited number of factors, and the second task is to ro¬ 
tate the arbitrary reference frame to that of a simple structure, if it can be 
found for some of these factors. In determining the first-factor matrix be¬ 
fore starting the rotation of axes, it is desirable that the given number of 
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factors sliall account for as much as possible of the variance of the correla¬ 
tion matnx If we could disiegard the practical pioblems of labor, time, and 
cost, we should analyze the correlation matrix by determining the successive 
principal axes with communalities m the diagonals These would be repre¬ 
sented by the successive columns of the factor matrix When the residuals 
became small enough to ignore, the factoring would be discontinued, and the 
factor matrix would then be ready for the rotational problem We should 
like to have the assuiance, if we could get it with a reasonable amount of 
labor, that the r factois that have been determined account for as much of 
the total variance as is possible with r factois That assurance would be rep¬ 
resented by the piincipal-axes solution for the first r principal factors But 
it should be noted that such a principal-factor matrix is not the final solu¬ 
tion if the purpose of the analysis is to discover the nature of the underlying 
order Such a principal-factor matrix is only the starting-point for the rota¬ 
tional pioblem All students of factor analysis aie theiefore interested m 
the pnncipal-faetoi solution, no mattei what their point of view may be 
as legaids the purposes of factor analysis Foitunately, this computational 
pioblem can be dodged by using one of the less laborious factoring methods 
to obtain a factor matrix with arbitrary oithogonal axes, continuing the 
factoring one or two factors further than would be required by the principal 
axes The rotational problem would then give essentially the same solution 
as would be obtained if the principal axes had been known No matter what 
method of factoring is used, it is a safe lule to continue the factoring until 
one is sure that the factoring has gone far enough Too many factors can do 
no harm, but too few factors are suie to cause tiouble in identifying the 
stiueture If too many factors aie determined in the factoi matux before 
rotation of axes, then the icsidual factois appeal m the lotation of axes, and 
they aie left without interpietation When the principal-axes solution be¬ 
comes available with less computational labor, it will, no doubt, be preferred 
by all students of this subject, and they will stait the lotational problem 
with the principal-factor matrix 

Another situation in which the principal factors are of interest is found 
in the lotational problem When the rotational problem begins, one can be 
leasonably sure that the test configuration is thin m one or more directions 
that represent residual factors which are to be ignored in the interpretation 
It would be useful to be able to deteimine the mean principal axis of the 
test configuration at the start of the rotational problem One of the residual 
factors would then be indicated The next puncipal axis might also be negli¬ 
gible as to variance In this way several lesidual factors might be deter¬ 
mined at the start of the rotational problem if the mean principal axes could 
be readily computed The residual planes can be identified m the rotational 
procedure, but some of this labor might be avoided if the mean principal 
axes could be readily determined. 







CHAPTER XXI 

THE APPRAISAL OF INDIVIDUAL ABILITIES 


In previous chapters the mam problem has been to devise factorial meth¬ 
ods which would be sufficiently powerful to deal with the complexities of 
multiple factors so as to reveal functional unities, with a minimum of restric¬ 
tion as to their nature and the interrelations of the factors To discover the 
factors and to sustain or refute postulated factors have been the principal 
purposes The development of the factorial methods has been a continuous 
process of compromising between the theoietically complete and ideal solu¬ 
tions and those solutions that can be made available with a leasonable 
amount of labor, time, and cost We turn, now, to the problem of apprais¬ 
ing individuals as to the factois that may have been identified, either as 
reasonably pure factois or at least as functional unities that aie practically 
useful in dealing with individual differences 

In dealing with this problem of appraising individuals as to their profiles 
of abilities, we shall develop, first, what seems now to be a rather complete 
statement of the solution for the factors that have been identified and which 
can be tested for, and then we shall consider the compromises that will be 
necessary m piactical situations which will raiely allow a complete solution 

The regression jtons 

Let us assume that a set of n tests is available for which the factonal com¬ 
positions are known and that the whole batteiy of tests is given to each sub¬ 
ject foi the purpose of wiitmg his profile of r factonal abilities Here, as be¬ 
fore, let the standard score of individual i in the pi unary ability p be de¬ 
noted x p , Similarly, let the standard score of individual i on test j be de¬ 
noted 8 lt The problem then is to estimate the standard factorial scores x pi 
in terms of the given and known test scores s„ This is a conventional form 
of statistical problem which calls for the regression x pi on s,, The given 
test scores are the independent variables in terms of which the individual 
factorial scores are to be estimated We then have the equation 


( 1 ) 


Xnt 


X 

j 


WjjjSji Gpi , 


where the subscript p refers to primary factors, j refers to tests, % refers to 
individuals, and w is the legression coefficient or weight of the score s,, m 
test 3 in the estimation of the primary factor p The residual or discrepancy 
between the true value of x pi and the best value that can be obtained as a 
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linear combination of test scoies is denoted e pt . It is assumed here that the 
true individual factor scores aic the values x Pt 

Equation (1) is a regiession equation, in which the test scores s ]t are the 
n independent variables and t pi is the dependent variable to be estimated by 
the best linear combination of the test scores The problem of finding the 
best linear combination of the test scores is that of finding the regression 
weights w vl for tests j in estimating the dependent variable which is the 
score x pi . in the primary factor p The weights w are to be so deteimined as 
to minimize the discrepancies. The discrepancies can be written by (1) ]n 
the matnx form 

(2) X - WS = E , 

where X of oidei ? XA is the matnx of factor scoies with elements x„„ W of 
Older rXn is the matnx of legrcssion weights iu p] , S of ordei nXN is the 
Standaid test scoie matrix with elements s,„ and E of Older rXN is the ma¬ 
trix of lesiduals in which the sum of the squaies of each row is minimized 
The problem can be solved separately foi each pnmaiy factor p. Then in 
equation (2) the matuces X, W/S, and E aie reduced to row vectors of older 
1 XN Then it is the n values of w PI in the i ow vectoi W of order lXn which 
are to be deteimined so as to minimize the lesiduals. * 

For each pnmaiy factoi p that is to be estimated in the subjects, we have 
i residual e P , for each individual These residuals aio the N elements in the 
low vectoi E The sum of the squares of these residuals is given by the ma¬ 
lm product EE', which is a scalai. Fiom (2) we have 

(3) EE' = (X - TIT)(A - If 'SY , 
which becomes 

(4) ICE' = XX' - XS'W - WSX' + WSS'W 
Since these terms represent scalars, we can wute 

(5) EE' = XX' - 2WSX' 4- IF AS''ll" . 

Fiom this we obtain the normal equation for IF, 

(C) X8' = TlhS'jS" 

Multiplying both sides by l/N, we have 



* H W Turnbull and A C Aitken, Canonical Matrices (London Blackie & Sons, 
1032), p 173 
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But 

(8) ^ XS' = R„, 

where ll v , is a row vector whose elements aie the correlations between the 
test scores s„ and the primary-factor scores x pi These are the coi relations 
between the tests and the primary factor p, and hence they are, in a sense, 
the validity coefficients of the tests foi the primary factor p Similarly, 

(9) jj SS' = R, k , 

where R Jk is the eonelation matnx with unit diagonals * Fiom (7), (8), and 

(9) we get 

( 10 ) R B k = W VJ R jk , 

and fiom this we have the desned equation, 

( 11 ) R„R7? = W,* , 

where w pk (or tv PJ ) are the desired regression coefficients in equation (1) 
Equation (11) has inteiestmg interpretations for statistical and psycho¬ 
logical problems In order to determine the legiession weights w, we need 
the inverse, RjT: This inverse does not exist if R,k is singular Hence the 
correlation matiix R,k with unit diagonals foi the independent vanables 
must be of lank equal to its older The numbei of tenns in IFS of the le- 
giession equation (2) is equal to the numbei of independent variables 
If the situation should arise m which a set of supposedly independent 
variables is not lineal ly independent, then the inverse, 7?,! 1 , could not be de- 
teimined In dealing with experimental data it may happen that the inde¬ 
pendent vanables owe their linear independence to fortuitous variable 
eirors If the variable errors weie then eliminated, the set of supposedly in¬ 
dependent vanables would no longer be linearly independent If, in spite of 
such a situation, the regression equation were computed with as many ic- 
gression weights as there are supposedly independent variables, we should 
expect to see absurdities m the i egression weights They would express only 
the fortuitous vanable errors This situation is not unlikely when a regres¬ 
sion equation is wntten in terms of a large number of independent variables 
The difficulty with correlated independent variables whose true rank may 
be less than the ordei can be avoided by an application of multiple-factor 
analysis The correlation matrix foi the independent variables should be 

* The unit diagonals are called for here because we are now dealing with a convention¬ 
al regression equation 
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factored with reliability coefficients in the diagonals These diagonals in¬ 
clude not only the common-factor variances of the independent variables 
but also those parts of the unique variances which the independent vanables 
may share with the dependent variable. These diagonals exclude, however, 
the unreliabilities of the independent variables When the residuals aio 
judged to be vanishingly small, the number of factors is also the numbei of 
terms which may be written legitimately in the regression equation 

Having determined the number of terms which can be wntten in the le- 
giession equation, one proceeds next to arrange the independent vanables 
in the same number of groups as there are number of teims in Wii, which is 
also the rank of the independent variables The gioupmg should be done so 
that closely correlated variables aie in the same group as far as possible A 
new set of independent variables can then be written as centroids of the 
gioups These, m turn, constitute the independent variables R jk in equation 
(11), with unit diagonals for deteimining the) weights w p u 

In application to a psychological problem, equation (11) gives results 
which are of theoretical, as well as of practical, interest Suppose that the 
number factor N is to be appraised by a large batteiy of tests which repie- 
sent several factors If the instructions for the numbei tests include some 
vaiiance on the verbal comprehension factoi V, then this factor will have 
small saturation on the number tests In writing the regression equation foi 
the appraisal of the number factor, one might find a small negative i egres¬ 
sion weight on the vei bal factor V The interpretation then would be that 
the number factor N is a linear combination of the number test composite 
N t and the verbal test composite V { , in which the numbei test composite 
N t is coplanai with N and V t and between them It is as if the numbei 
factor N would be determined from N and V by subtracting a collection for 
the veibal component in the numbei tests. This is illustrative of the system 
of weights that may arise in a complete solution foi primary-factor scores 
as linear combinations of the tests m a diversified battery. 

Much thought has been given by some students of factorial theory to this 
problem of appraising the primary-factor scoies of individuals on the basis of 
their scores in a diversified test batteiy The solution to this problem is giv¬ 
en by the regression equation (1) and equation (11) for the regression coeffi¬ 
cients, but m practical application it is rarely feasible or even desirable to 
use the theoretically complete solution 

The regression equation (1), or its valiants, presupposes that a complete 
test battery has been given to each subject, and it implies that the battery 
represents all the relevant pnmaiy factors. In practice it is desirable that 
the tests for each factor be as far as possible self-contained, so that a subject 

* This is the problem which was discussed in a different setting by Ragnar Frisch m 
his monograph, Statistical Confluence Analysis by Means of Complete Regression Systems 
(Pub No 5 [University of Oslo Economic Institute, 1934]) 
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can. be (examined for one factor ■without corrections for the other factors 
which may be n i elevant for the occasion In order to prepai e tests to satisfy 
this requirement we merely select several tests which have high saturation 
on the desiied factoi, and the scoie in such a composite may then be used m 
pi actical situations as the subject’s factor score The additional lefinement 
that could be bi ought into the appraisal by giving the tests foi all othei 
known factors m oidei that a regression equation might be solved for each 
individual for each factor is 1 arely, if ever, worth the extra trouble In most 
situations this complete procedure would even be absuid 

In selecting the subtests which are to constitute a composite test for any 
primary factoi, it is useful to distribute as far as possible the variances on 
the factors which are melevant for the composite Thus in subtests for a 
composite to appiaise the space factor S, the seveial subtests would prob¬ 
ably be found to be unpuie, in that each of them would have some variance 
attnbutable to veibal compichension V in the test instruction, to the num¬ 
ber factoi N if a pait of the task induces numerical thinking, and so on 
Now, if all the space tests had an appreciable secondary saturation on the 
verbal factoi V, then evidently the composite would represent a lineai com¬ 
bination of S and V with major variance in S This effect is reduced by se¬ 
lecting the subtests so that the 11 relevant variance is scattered among differ¬ 
ent factors 

When a test composite for a primary factor has been selected, it can be 
used as a test for the factor in question quite independently of test com¬ 
posites for other pumary factors that may or may not be given to the same 
subjects It should be specially noted that the conelations between the test 
composites for the seveial primary factors will generally have highei mter- 
correlations than the primaiy factors themselves Fuithei, if the primary- 
factoi scores aie estimated by the legression equation (1) for a factonally 
diversified test batteiy, the correlations between such factor scoies may be 
expected to be lowci than the conelations between the corresponding test 
composites Nevertheless, the problem of appraising an individual as to 
each of several factors is best solved in the practical situation by using a test 
composite for each factor In theoretical studies of the intrinsic conelations 
between the factois it is preferable to use the more complete factonal meth¬ 
ods previously described 

We turn next to the othei regression, namely, to predict the test scores 
Sj, when the individual factoi scores x pi are known 

The regression soar 

This 1 egression implies that the primary abilities of an individual are 
known and that it is desired to estimate what his performance will be on a 
test with known factorial weightings This is the reverse of the pievious re- 
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gression x on s, in which it was assumed that an individual’s scores are 
known and that his primary abilities are to be appraised 
The case m which s„ is to be estimated by t p , can be written m the form 

(12) Sjj = WjpZpt A £}i , 

V 

in which the notation is the same as in the pievious case. By similar reason¬ 
ing we get, for the regiession coefficients, 

(13) — R]j)R p q 

The matrix R JP is of order nXr, and it shows the coirelation between each 
test j and each primary factor p These correlations are, in a sense, validity 
coefficients, because these are the best available conelations between the 
tests and the factors that the tests are intended to appraise The matrix 
Bp, 1 is of order rXr, and it is detei mined from the mtercorrelations R Pt . The 
matrix R P , has unit diagonals because it shows correlations between factors 
of unit variance m the common-factor space Since the pumary abilities are 
linearly independent, it follows that the lank of is r and hence that R pq 
is non-smgular 

If the primary abilities are uncorrelated, the matrix R rq becomes an 
identity matrix and its inverse is then also the identity matrix The regres¬ 
sion weights then become 

(14) w )P - R 3P = F (oithogonal case) , 

and in this case the legression weights are, in fact, the elements of the factor 
matrix F, whose elements show the correlations between the tests and the 
orthogonal unit refei ence factors The corresponding i egression equation for 
the simplest case m which all the contributing factois are common then is 

(15) s 3 , == F 3p x p , (orthogonal case), 

which is also the fundamental observation equation with which we staited 
the development of multiple-factoi theory 
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test,/ and single-common factor, 203 
tests and general factor, 422 
detei miners of, 325 
effect of selection on, 444-50, 454—69 
multiple, 59-60, 68, 179, 327, 504, 511— 
16 

negative, 91, 93, 99 
notation for, 86 
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pumaiy vectois, 364 
reference vectois, 136, 204, 226-28, 
241, 251 

test vector and reference vectoi, 132 
direction, see Direction cosines 
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vanable, in selection, 443-53, 458, 464, 
470 

Crossroads m the Mind of Man, by Tru¬ 
man L Kelley, ix, 293, 309 


Curve-fitting, 368 
Cylmdei problem, 117-124 

Data 

chaiactensties of, 375-76 
of factor analysis, 57, 63, 101 
factorial interpretation of, 378 
fallible, 101, 260-61, 300-301, 505, 508 
treatment of, 177, 375-76 
Degrees of fieedom, 52 

Dependence (see also Independence) 
expeumental, 441-42 
linear, see Linear dependence 
Denvativea, partial, 322, 475 
Designs test, 366 
Detei mutant 

chaiactenstic, 27-28, 476-77 

cofactor of, 7-8 

of composite matrix, 302-4 

defined, 5 

diagonals of, 3 

evaluation of, 6-9, 11-13 

of hierarchy, 290-91 

of a matrix, 3-13 

minors of, 6-7 

order of, 3 

of product of two matrices, 20 
opei ties of, 3-13 
term of, 4-5 

of transformation matux, 40, 479-80 
Diagonal 
principal, 3 
secondary, 3 
Diagonal entries 

oommunahties as, see Coimnunality 
m correlation matux of unit rank, 259, 
262, 274, 277-81, 284 
criterion for determination of, 284, 287 
geometrical interrelation of, 137-38 
in Heywood case, 289-90 
in Hotelling’s iterative method of fac¬ 
toring, 484 
imaginary, 308 
in Kelley’s work, vm 
known, 101-5, 170-75 
positive, 290 

in principal-axes solution, 177—78, 484, 
509 

and lank, 82, 282-87 
leliability coefficients as, 83 
in Spearman's single-factor method, 274, 
284 
size of 

greater than coinmunalities, 286 
gi eater than unity, 262, 289—90 
high and low, 284-87 
less than communalities, 286 
less than unity, 284, 286 
unity, 98, 176-77, 284-87, 291, 484 
as stretching factors, 132 
in tetrad difference equation, 284 
unique factoi contribution to, 79, 320 
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Diagonal entries —continued 
unknown, 110-16, 149-67, 283 
variable, 287 
Diagonal matnx 
defined, 21 
types of, 21-23 
use of, 132, 137-38, 351-53 
Diagonal method of factoung, 101-5, 175, 
308 

Dickson, L E , 1, 44 

Dimensionality {see also Rank) 
augmented, 421-22 
of configuiation, 91-92, 194 
of first-order domain, 411—14 
loss of, 204, 239-40 
of second-older domain, 411-21 
of total test space, 176, 448 
Direction cosines 
defined, 34 
of normal 

to hyperpiano, 34—35, 226-27, 238-39, 
349-51 

to score suiface, 323-24 
positive, 216, 342 
in positive legion, 342 
of pi unary vectors, 136-38, 228, 351 
of principal axes, 477-79 
pioperty of, 35 

of refeience vectois, 38-43, 132-38, 195- 
205, 227, 347, 384, 474-77 
as scale in plotting configuration on 
sphere, 132-37 

in transformation matrix, 38-43 
of unit test vectors, 137, 349 
Direction numbeis, 50, 227, 383-85, 398, 
474 

Dispersion, measures of, 368 
Distribution of ability, 62-66, 325, 307-69 
Disturbers of tetrad equation, 273, 473 
Domain 

alternative simple structuics m same, 
329—31 

defined, 55, 320 

of factor analysis, 55-57, 324-25, 340, 
375-76 

paiameteis of, 71-72, 177-81, 313-14, 
320, 413 

relation between first- and second-ordei, 
411-34 

of statistical method, 291 
underlying order in, 55-56, 258, 320-25, 
360, 472, 506-7 

Double-gioup method of rotation, 408-10 
Doublet 
defined, 182 

m estimation of communalities, 299, 
306-7, 315 

occunence of, 363, 441-42, 468 
&econd-oi dei, 417-18 
Dleaden Arnold, 1 


Elements 

of ooiielation matrix, 71-73, see also 
Correlation coefficients 
of diagonal matnx, 21-23 
of factoi matrix, 71-73, see also Factor 
loadings 
of matrix, 2 
notation foi, 138 
of population matnx, 08, 71, 356 
of score matnx, 68 
umtaiy, 343-44 

Ellipsoid, piojoction of test vectois into 
444-45, 448 
Equation 

charactenstic, see Chaiactenstic equa¬ 
tion 

fundamental factoi theoiem, vi, 77-81 
149 ’ 

fundamental obseivation, 68, 71, 325 
357-59 

giaphical lepresentation of, 14-18 
of nyperplane, 34-35 
of line, 33-35, 499 
lineal, 33-38 

for new communalities after selection 
450-52 

foi new correlations after selection, 448- 
50, 452, 455 

normal form of, 34-35, 226, 475 
notation for 

expanded, 13, 16 
matnx, 15-19 
lectangulai, 14-15, 17-18 
summational, 15-16, 28-31 
of oblique simple stiueture, 354-56 
Otis-Kelley foimula, 449-50 
paitial corielation foimula, 448-50 
of plane, 34-35 
rational, 52 
regiession 

s on v, 515-16 
x on s, 511-15 
for scoie surfaces, 322 
simultaneous, 13-18, 291 
of slope of line, 33, 499-500 
of test vectoi, 148 
tetrad-difference, 265, 273, 293 
of trace, 226-27, 241 
vector, 234, 239 
Eri oi 

experimental, 146 
landom, 73, 83, 261 
sampling, x, 260-61, 264, 283 
standaia, 261, 267 
uncoil elated, 73, 83 

variable, 73, 83, 286, 306-7, 505, 508, 
613 

Eli oi factor, 73-75, 83-86, 146 
Biror-factoi vanance 

and conection for attenuation, 146 
notation for, 85-86 

occunence of, 73-75, 83-85, 286, 505, 
508 
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and lehabihty, 84 

separation of specific factors from, 73, 
84, 286 

Essential Tiails of Mental Life, by Till¬ 
man L Kelley, vm, 287 
Experimental dependence, 441-42 

Factor analysis 

assumptions in, see Assumptions 
British school of, 273 
data of, 57, 63, 375-76 
and distribution of ability, 62-67, 325, 
367-69 

criteria of acceptability in, see Cnteua 
domain of, 55-57, 313-14, 324-25, 375- 
76 

and empirical prediction, 59-62, 2S7, 
332 

exploratory nature of, 56-57, 256, 269, 
287, 320-27, 332-34, 340, 365 
and faculty psychology, x, 56, 70, 145- 
46, 177 

first approximation in, 54-55 
fundamental concepts in, 328-29 
fundamental equations in, 68-86, 149 
fundamental postulates of, 62-63 
ideal constructs in, 70 
ideal solution in, 179 
methods of; see Faetonng methods, Ro¬ 
tation of axes 

and multiple oouelation, 59-62, 68, 179, 
327 

parameters of. see Parameteis 
planning of, 176, 313-14, 375-76 
purpose of, 55, 62, 70, 92, 149, 176-81, 
325, 340, 348, 360, 375, 471-72, 503-8 
as a scientific method, 319, 338 
steps m, 1.01, 283 
texts in, v, vm-ix 

uniqueness of solution in, 92-93, 178, 
180, 329-34 

Factor Analysis, by Karl J Holzingei and 
Harry H Harman, vm 

Factor Analysis to 1940, by Dael L Wolfle, 

IX 

Factor axis, 120-21, 473, see also Reference 
frame 

Factor loadings 
augmented, 131 

centroid, 152-54, 157-59, 161-65, 167 
170, 279 

common-, 73—86, 146 
contribution to variance of, 68—86 
defined, 71, 73 
effect of 

adjustment of planes on, 249-51 
selection on, 360, 369 
error, 86, 146 
extended, 225 

geometrical interpretation of, 89, 99 
in Heywood case, 289-90 
invariance of, see Invariance 
maximum variation in, 362 


negative, 96, 365, 368 
notation for, 86 

number of linearly independent, 292 
in principal-axes solution, 149, 473-91, 

as projections on eo-ordinatc axes, 89, 
281 

reference abilities underlying, 68-69, 325 
as regression coefficients, 179, 504 
relation to correlation matrix, 95, 100, 
281 

single-common, 259-65, 270, 273-76, 
279-81, 284, 289-90 
in Spearman's formula, 263 
specific, 86, 146 
unique, 78, 86 

zero, 89, 147, 181, 292, 324, 340-41, 
355-56 

Factor matrix 

augmented oblique, 422-27 
centroid, 167, 170, 194, 231, 285-87 
common factors in, 73-86 
complete, 76-82, 86, 146 
consistent, 289-90 
effect on 

of eommunahties, 282, 312 
of complexity, 326 
of factoring method, 363 
of selection, 455, 468-69 
of tieatment of scores, 368-69 
entiles of, 74-76, 87-89, 147, 181-91, 
292, 326, 334-46, 349, 377-78 
eiioi factors in, 73-86 
extended, 225-53 

factor patterns in, see Factoi patterns 
fundamental equations for, 68, 71-73, 
76-82 

geometncal intei jrretation of, 87—100 
imaginanes m, 286 

interpretation of, 95-96, 99, 177, 254, 
334-35, 349, 365 
invariance m, 363-65 
normalized, 131-33, 146 
notation for, 86 

number of independent parameters in, 
177, 291-94, 307, 311, 335 
order of appearance of primary factors 
in, 255 

for principal-axes solution, 477-92, 496- 
502, 509 

rank of, 02, 97-98, 343 
reduced, 76, 79-82, 86 
reduction of, to unit rank, 268 
leference frame of, see Reference frame 
relation of 

to correlation matrix, 79-82, 92, 95, 
100, 281 

to factorial structure, 92 
to population matrix, 71 
rotated, 135-39, 202-6, 210-13, 228-29, 
253, 324, 326, 332, 334, 347-53, 369, 
378 

row of, 87 

second-order, 419 

sign reversals ill, 96-97, 100 
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Factor matrix— continued 
and simple structure, 341—43 
single-common, 250-62, 268, 28G, 288- 
90, 488 

specific factors in, 73-80 
test coefficients in, 71-73 
transformation from oithogonal to 
oblique, 347-54 
transpose of, 78-81 
unique factors in, 73-70, 79 
uniqueness of, 93, 289-90, 343 
Factor patterns 

characteristics of, 191-93 
corresponding to eonfiguiational types, 
88, 97, 182-90, 419, 441, 443 
defined, 191, 320 
effect of scoie treatment on, 309 
first- and second-order, 419 
role of, xi, 365 

in simple structure, 181-82, 190, 369 
Factor problem 

deterininacy of, 81, 361, 378, 473 
formulation of, xi, 76 
multiple-, 283, 473 
non-linear, 510 
single-, 259-65 
starting-point of, 55, 71, 92 
steps in, 101 
types of, 340 

Factorial Analysis of Human Ability, The, 
by Godfrey Thomson, ix, 283, 290, 
448, 464, 465 

Factorial invariance, see Invariance 
Factorial Studies of Intelligence, by L L 
Thuistone and Thelma Gwinn Thur- 
stone, 402 

Factoring methods 
applicability of, 176 

Burt’s foimula for con elation hier¬ 
archy, 279 

centroid, 111, 149-75 
centroid versus principal-axes solution, 
178 

criterion of validity of, 281, 375 
diagonal, 101-7 
direct summation, 277-78 
graphical aids to, 497-500 
group centroid, 157-60 
grouping, 111-16 

iterative, of Hotelling, 279, 483-85, 497, 
503, 505 

logarithmic solution, 277 
multiple group, 170-75 
simplified centroid, 159 
Spearman's smgle-factor formula, 273- 
76 

successive ap 2 iroxiinations, 278-79 
tetrad diffei cnees procedure, 265-68 
for unit rank, 273-79 
Factois 
average, 180 

bipolar, 121, 182, 189-90, 193, 215, 363 


causative, 435 
central, 180 
centroid, 152-53, 232 
common; see Common factors 
contribution to variance of, 68-86,340-41 
dependent composite, 190-91 
doublet, 182, 299, 306-7, 315, 363, 417 - 
18, 441-42 
dynamic, 345-46 
enor, 73-75, 83-86, 146 
fust-order, 411-39 

geneinl, see Geneial factoi; Beoond- 
ordei factois 
gioup, see Gioup factois 
lughei-ordei, 414 
imaginary, 286 
incidental, 436-42, 469, 471 
linear combination of, 68 
natuie of, 55—59 
orthogonal, 70, 73 
oithogonal geneial, 416 
primary, see Pumary factois 
principal, 478, 509 
reference, 71-73, 85, 99 
relation to vaimnce of, 73 
second-oider, see Seeond-oider factois 
secondaiy, 273 
specific, see Specific factoi b 
static, 345 
thud-order, 414 
tnplet, 182 
types of, 182 
uncorrelated, 139-40, 411 
unique, see Unique factois 
unitary, 343-44 

Factois of Mind,, The, by Cynl Buit, ix 
Faculty psychology, x, 56, 70, 145-46, 177 
Fonai, W I,, 1 

Fust Couise m Theory of Equations, by 
L E Dickson, 44 
Flywheel pi oblem, 344-45 
Fnscb, Hagnar, 514 

Functional unities, 57-58, 145-46, 284-85, 
506—8, see also Piunaiy factois 
Functions 

as analytical cntei la foi reference fi ame, 
377-78 

cognitive, 333 
linear, 54-55 
monotonic, 322, 368-69 
of paimneteis of factorial domain, 322- 
23 

some, 319-27, 365, 368-72 
Fundamental concepts, 328-29 
Fundamental criterion of valid factoi ini 
method, 361 

Fundamental equations, 68-86, 149 
Fundamental factor theorem, vi, 79-81, 
92, 149, 170, 259, 351, 353-54. 359, 
504-5 

Fundamental postulate, 63 
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n-factor, vi, vm, 188, 271-73, 284, 421, 
439, 507 

Gaussian 'distribution, 64-66 
General factoi 

configurations involving, 188-89 
defined, 121, 182 

differentiated fiom general common fac¬ 
tor, 130-31 

orthogonal, 271, 273, 416-19 
second-order, see Second-Older factors 
Spearman’s g, vi, vm, 188, 271-73, 284, 
421, 439, 507 

Geneial intellective factoi g, see Geneial 
factor 

Geneiatmg paiameteis, 434-37 
Genetic basis of mental traits, 333 
Geometrical interpretation of 
augmented co-ordinates, 349 
communality, 87-100, 120, 282 
correction for uniqueness, 146-47 
correlation coefficients, 89-91 
correlation matnx, 91-96, 99, 145-48, 
282 

factor loading, 89, 99 
factor matrix, 87-100 
lineal equations, 33-38 
matrices, 36-37 

numbei of tests to determine single 
common factor, 264-65 
primary factors, 349-52 
? <n common factors, 97-98 
reference factoi, 99 
reflection of tiaits, 153-57 
residuals, 153-57 
score functions, 321-24 
sign leversafe in the factor matrix, 96- 
98, 10U 

simple structure, 96, 99, 130, 257-58, 
354^55 

single-common-factor loading, 270 
test battery, 89 
tests, 87, 99 
transformation 
oblique, 43 
orthogonal, 38-43 
zero factor loadings, 89, 135-36 
Geometrical models 
three-dimensional, 125-48 
two-dimensional, 87-100 
Gottschaldt Figuies test, 366 
Giamian matrix, 10, 304 

Giamian piopeitics of con elation matnx, 
82, 282-83, 286, 289, 304 
Graphical methods 
as aids to factoring, 497-500 
determination of hyperplane, 377-78 
determination of two planes, 238 
estimating communnlities, 306 
inspection of constellations, 344 
isolation of simple structure, 349—50 


iterative determination of principal 
axes, 497-500 

locating reference flame, 225-58, 377 
plotting of 

normalized vectors on sphere, 131-39, 

three-dimensional sections, 195, 225- 
39 

two-dimensional sections, 194-224 
rotation of axes, 131-39, 205 

Giaphical representation of 
equation, 14-18 
factorial structure, 99 
piopoitional columns, 277 
simple sti ucture, 257-58 
test configuration, 125-39, 147-48 
tetrads, 270-72 
Graustem, VV C , 1, 41 
Great circles, 132, 134, 349 

Gioup centroid method of factoi ing, 157- 
61 

Gioup factors 

configuiations involving, 188-89 
defined, 182, 421 
as disturbers, 273, 473 
and primary factoi s, 420-21 
projection of tests on, 422 
Gioupmg method of factoring, 111-16, 149 

Hannan, Harry H , vm 
Heywood, H B , 289 
Ileywood case, 289-91 
Hieiarchy, 273, 284, 289-91, 487 
Holzinger, Kail J , vm, 171, 267 
Hotelling, Harold, \u, 279, 482-91, 493- 
97, 503, 505-6 
Hyperplanes 

angular separation of, 349 
bounding, 343 

co-oi chnate, 327-28, 347-49, 354-55, 
382-84 

defined, 328, 342—43 
dimensionality of, 34-35, 225, 328 
equation of, 34-35, 354 
mteipretation of, 337 
normals to, 34-39, 335 
oblique, 349, 354-55 
orthogonal, 41, 349 

oveidetermination of, 216, 336, 338, 
349-51 
positive, 342 

positive co-oidmate, 343, 384 
secondary, 331 

in simple structure, 327-28, 331, 335, 
342-43, 354-55 

for the trait configuration, 327—28, 354— 
55 

Hypotheses in factor analysis, 122-23, 
187-88, 216, 256, 291, 324-26. 344, 
506-8 



526 


MULTIPLE-FACTOR ANALYSIS 


Identity matrix, 22—23, 78, 410 
Independence 
defined, 63 

experimental, 03, 21)1, 313 
linear, 63; see also Linear dependence 
statistical, 63, 126 
Individual diffeieneeh 

abilities as causes of, 52, 69-70, 325 
and identical test scores, 70 
paiameteis involved m, 58-59, 324-25 
at point age, 333-34 
study of, 52, 55, 333-34 
Intelligence, theories of, 439 
Intercolumnar conelation ci'itenon, 2C0- 
62 

Interoolumnai pioportionnlifcy enterion, 
260-62 

Intel correlations, see Correlation c'ef¬ 
ficients, Coi relations 
Interpretation 

of geneial intellective factoi g, 273 
objective, 145, 338 
plausibility of, 336-40 
of primary factors; see Pinnaiy factois 
of reduction to unit rank, 268-70 
of second-older factors, 414 
subjective, 145, 338, 506-8 
of tiait configuration, 327 
Interval, scoie, 367 

Introduction to Algebiaic Theories, by A A 
Albeit, 1 

Introduction to Higher Algebia, by Maxune 
B6cher, 1 

Introspection m factorial investigation, 
326 

Invariance 

configurational, vm-ix, 364-66 

criterion of, 360 

of factoi loadings, 360-76 

of factor pattern, 191 

factorial, 360-76 

metric, 363-64 

numerical, vm, 363-64 

of scalar pioduct, 91 

of subbatteiy of tests, 363, 373-76 

types of, 363-67 

of weights of primary abilities, 361 
Inverse 

computation of, 25-26, 46-48 
of a matrix, 23—26 
of pioduct of matnces, 25 
in regression equations, 513, 516 
of transfoimation matrix, 137 
Isolation of 

common factoi s, 73-70, 81 
primary abilities, 73, 361, 506-8 
simple stiucture; see Simple stnictuie 
specific factors, 73-76 
unique factois, 73-76 
Iterative method of factoring, 279, 482- 
500, 503, 505-6 


Kelley, Tinman L , vm, 287, 293, 300 
Kelley-Wood tables, 06, 368 
Ivioneckei’s delta, 39 

Lagrange’s method of undetermined multi 
plieis, 475 

Law, scientific, 51-55 
Least-squares problem, 149, 504, 508 
Ledeimann, Walter, 287-89, 444, 448 
Tune 

equations of, 33-34 
slope of, 33, 218-20, 499-500 
Linear combination of 

columns of con elation matrix, 95, 281 
factors, 128-30, 147-48, 352-53, 413-14, 
504 

test seoies, 511-12 
tests, 69, 440-43 
Lineal dependence of 

arrays of matrix, 32-33, 300-301, 314-18 
centioid vectors, 171 
columns of factor matux, 82, 204, 291- 
94 

factors, 63, 60, 82, 282, 412-22 
planes, 238-39 
primary vectois, 420-22 
reference vectois, 328 
tests, 335 

vectois, 48-50, 239 
Lineal equations, 33-38 
Lineal functions, xi, 32-33, 54-55, 63, 60 
Loadings; see Factor loadings 

Logarithmic solution of single-factor prob¬ 
lem, 277 

Long refeienee vectors, 196-202, 234-38 
Matrix 

adjoint, of, 9, 25 
array of, 2, 14 
cell of, 2 

characteristic, 27-28, 476 
column of, 2 
composite, 301-4 

con elation, see Correlation matrix 
covariance, 453, 455 
defined, 1 

determinant of, 3-13, 20 
diagonal, 21-23, 132-38, 351-53 
difference of two, 19 
element of, 2 
equation, 15-19 
factoi, see Factor matrix 
of factoi seoies, 512 
formulation of factoi problem, ix, 76, 
250-62 

geometrical interpietation of, 36-37 
Gramian, 10 

in Holzmgei’s method, vm 
identity, 22 

inverse of, 23-26, 46-48 
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multiplication, 14-21 
associative, 19 
ci oss products in, 14 

distributive, 20 
non-commutative, 18 
post-, 19-20 
p re -, 19-20 

summational notation in, 29—31 
non-singular, 11 
order of, 1-2 

orthogonal by columns, 41 
orthogonal by rows, 41 
population, 68, 71, 86 
positive-definite, 10 
rank of, 10 33 
reciprocal of, 

of regression weights, 512 
row of, 2 

ITX'm, 71, 77 86 93 324, 504-5 
sectioning of, 171, 301 4 
singular, 11 
skew symmetuc, iu 

sum of two, 19 
symmetric, 10, 46, 473 
of a transfoimation, 33-43 
transpose of, 3 
transpose of product of, 16 
Matt lx-multiplying machine, 206 
Maturation factor, 507 
Maximum variance cntenon, 177-78 


Multiple gioup method of factoring, 170- 
75 

Music factoi, 287 

Noimal distribution of ability, 62-63, 325, 
367 

Normal to hyperplane 

direction cosines of, 34—35, 226—27, 238- 
39, 349-51 

lelation to pnmary trait vectors, 347- 
52 

and zeio factor loadings, 323-24 
Notation 

for equations 

expanded, 13, 16 
matrix, 15-19 
rectangular, 14-15, 17-18 
summational, 15-16, 28-31 
for primary vectors, 348 
in scientific research, 332 
summary of factorial, 85-86 
Number of 

factors determined from n tests, 291-94 
factors veisus number of tests, 82, 97-98, 
291-94, 307-11, 440 
tests to determine r factors, 291-94 
tests to determine single-common factor, 
262-65 

tetrads for n tests, 265-70 
Number factor, 73, 145, 333, 514-15 


Mean of test scoies, 323-24, 368 
Measurements, see Scores 
Medland, Frank, 318 
Memory factor, 145 
Mental age scoi e, 442 

Method of extended vectors, 225-58, 406, 
458 

Method of minimizing weighted sums, 
396-408 

Method of undetermined multipliers, 475 

in estimation of communalities, 296-97, 
304, 308-9 

in evaluation of characteristic equation, 
27-28, 476 
first, 7 

and ideal rank of R, 317-18 
principal, 6, 262, 286, 304 
second-order, vi, vm, 265 68, 2iu li¬ 
as tetrads, 265-68 
for unit rank, 265-68, 270 
Models 

geometrical, 87-100 
spherical, 127-43 
vector, 125-27 

Modern Algebraic Theories, by L E. Dick¬ 
son, 1 

Multiple correlation, 59-60, 68, 179, 327, 
604, 511-16 


Observation equation, 68, 71, 325, 357—59 
Octants, 132, 147-48 
Opposites test, 360 
Order 

concepts of underlying, 56—56, 58, 258, 
320-38, 436 
defined, 328 
of determinant, 3 
hierarchical, 273 
of matrix, 1-2 
oblique simple, 321 
orthogonal simple, 321, 328 
and rank, 82, 282, 286, 484 
simple, 320-21, 328—29 
Orthogonality 
of matrix, 41 
lestuction of, vn, 139^0 
of test vectors, 91 
and zeio factor loadings, 89-91 
Osgood, W. F , 1, 41 
Otis-Kelley formula, 449-50 

Paiallel testa, 83 
Parameters 

of abilities, 58-59, 319-27 
alternative, 331-34 
as causative factors, 434-65 
choice of, xi, 333-34 
complexity as number ol, 320 
correlated, 139—40, 320-21 
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Parameters —continued 

m correlation matnx, 291-04, 307 
defined, 331, 414 
and degiees of freedom, 52 
of domain, 180, 291-92, 320, 331-33, 
471-72 

m factor matrix, 177, 291-94, 307, 311 
generating, 434-37 

geometncal representation of, 127, 319- 
25 

identification of, 176-77, 324, 365 
incidental, 436-39 
linear dependence in, 412-16 
in linear equations, 34-35 
primary factors as, see Primaly factors 
second-order, 411-16 
of set of scoies, 319—25 
uncorrelated, 139-40, 320-21 
unique, 320, 332-33, 412 
Partial correlation foi mula, 448-49 
Percentile lanks, 64-66, 368 
Personality schedules, 322 
Personality traits, 322-24 
Plane and Solid Analytic Oeomelry , by 
W P Osgood and W. C Graustein, 1, 
41 
Planes 

adjustment of, m rotation, 241-51 
best-fitting, foi score sru faces, 323-24 
check on independence of, 238-39 
co-ordinate, 181, 225-28 
equation of, 34-35 
trace of, see Trace 
Plotting of 

normalised vectois on a spheie, 131-39, 
194 

oblique lefeience axes, 206 
three-dimensional sections, 195, 226-30 
two-dimensional sections, 194—224 

Polynomial, loots of, 44-45 
Population 

equivalent, 360-61 

experimental, 55, 62, 324-26, 367, 458 
general, 325, 367 

selection of, 55, 324-25, 360-61, 438, 
440, 457-58, 470-72 
statistical, 62 
Population matrix 

for common factors, 71, 356 

for coirelated pnmary factors, 356-59 

elements of, 68, 71, 356 

notation for, 86 

oblique, 356-59 

relation of 

to factor matnx, 71, 77-78 
to acme matrix, 71 
transpose of, 77-78 

Positive manifold, 215, 217, 341-43, 397 
Positive octant, 132, 147-48 
Positive region, 342 


Piediotion 

factor analysis and empirical, 59-62 rr 
179, 501, 511-16 ’ ’ 

of factorial composition of new tot* 
333, 366-67, 506 

and postulated factors, 338, 366-67 

Primary abilities (see also Abilities: Pri¬ 
mary factors) 

ciiteuon for valid method of isolating, 
361 

invariance of weights of 361 
isolation of, 73, 361, 506-8 
in single-factor matrix, 268 

Piimaiy factois (see also Common factors) 
appraisal of individuals as to, 62-63 
325-27, 511-16 
and constellations, 344—45 
eon elation between, 78, 137-44, 212-15 
228-29, 320, 351-59, 411-39, 457-58 
470 

correlation of test with, 327 
cnteua for acceptability of, 336-40 
defined, 348 

effect ot selection on, 456-64, 470 
exploratory studies of, 55-59, 122, 255- 
5b, 287, 319-21, 333-34, 340, 364-67 
503-8 

fundamental factoi theorem for, 77-81 
353-54 

genetic basis of, 333-34 
and gioup factois, 420-21 
incidental, 436-39 
intci pretation of, 
geometrical, 349-52 
psychological, ix-x, 56, 98, 122-24, 
138-41, 144-46, 256, 268, 333, 337- 
40, 378, 430-40, 470-71 
invariance of, 333, 360-70 
isolation of, 55-67, 134, 179-81, 325, 
338, 347-50 

lineally independent, and lank, 82, 282 
naming of, 56, 145 
number N, 73, 145, 333, 514-15 
oblique, 63, 354 

order of appearance in factoi matnx of, 
255 

orthogonal, 63, 353 

as paiameterg, ix, 55-59, 332-33, 471- 
72 

positive, 215-16 

piedictability of postulated, 338 
prediction of scoies on, 511-15 
relation of, to factor matnx, 71-73, 334- 
36, 347-54 

and second-order factors, 411-39 
space S, 145, 324-26, 515 
thorny of, 55-67, 319-29, 345-46 
uncorrelatod, 349, 411, 416 
uniqueness of, 329, 332-35, 469 
unitary, 344-45 
validity of, 339-40, 472 
veibal V, 145, 333, 338, 361-62, 508, 
514-15 

word fluency \V, 73, 338, 508 
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Primary trait vectors, 348-54 

Primary traits, 347-52, see also Primary 
factors 

Primary vectors 

correlated, 411, 421 
cosines of, 137, 411 
defined, 137, 348 
direction cosines of, 137 
notation foi, 348 
relation of 

to correlations, 353-54 
to group factois, 420-21 
to hyperplanes, 347-49 
to test vectors, 352-53 
in second-order domain, 422 
of unit rank, 421 

Principal axes 
defined, 478 

direction cosines of, 474, 477-79 
location of, 177-78, 3G2, 473-77, 480- 
,500 

major, 286, 362-63, 415, 474, 478, 486- 
87, 503, 505-6, 508 
mean, 478, 487, 503, 509 
oithogonal, 477, 508 
as reference frame, 177-78, 284-87, 362, 
473-510 

l elation of, to centroid axes, 149, 177- 
79, 225, 362 

transformation to, from orthogonal ref¬ 
erence fiame, 474-77, 506 

Principal-axes solution 
advantages of, 503—10 
applicability of, 284-87, 296, 474, 503- 
10 

and centroid method, 177-79 
charactenstic equation m, 44-45, 476- 
79, 483-84, 487-91, 494-97, 500-503 
characteustics of, 177—78, 362, 473-74 
defined, 474 

diagonal entnes in, 474, 484, 509 
direct, viu, 480-97 
factor loadings m, 149, 473-91, 497 
factonal interpretation of, 216, 362 
graphical aids to, 497—500 
of Hotelling, 279, 482-500, 503, 505-6 
induect, 473-80, 484 
iterative method in, 297, 482—500, 503, 
505-6 

of Kelley, vm 

as least-squaies solution, 149 
limitations of, 216, 503-10 
maximizing of factor loadings in, 149 
maxmnum-vauance cutenon m, 177 
numerical example of 
direct, 484-85, 491 
indirect, 477-80 
properties of, 500-503 
rank m, 484 
residuals in, 493-97 
samplmg-erroi' formulae foi, x 
simple-structuie solution veisus, 503 
squaring process m, 485-92 


theoiems pertaining to, 500-503 
theory of, 474-77 
uniqueness of, 178, 474, 503 
Principal components 

of Hotelling, 482, 484, 486 
Tucker’s method of determining, 497 
Probability cuive, 66 
Piobability tables, 368 
Profile of ability, 511 
Projections of test vectois 

on centroid axes, 151, 217-19, 225, 251 
297-98, 362 

as check of independence of planes, 238- 
defined, 94 

into an ellipsoid, 444-45, 448 
on group factor, 422 
maximizing sums of squares of, 473-74 
minimizing weighted sums of, 377, 396- 
408 

normalized, 131-39 
numerical invariance of, 363 
on principal axes. 362-63, 473-91 
properties of, 89, 92-96 
on lefeience axes, 89, 92, 196-97 
on second order factoi, 422 
on single-factor axis, 276-79 
zeio, 197, 324 
Piopoitionality 

of arrays ol matnx, 32-33 
of columns of factor matrix, 269 
m correlation matnx, 260-62 
criterion of mt.ei columnar, 260-62 
of two variables, 291 
Psychology, science of, 52, 338 

Quaitile lange, 368 

Kandom contigiuation, 182-83 
Rank 

of correlation matrix 
complete, 82, 282-85 
reduced, 82, 91-92, 282-83, 308, 411- 
14, 456, 470-71, 473, 484, 503 
defined, 10, 282 
effect on 

of diagonal values, 282-87, 296-307, 
484 

of modifying law scores, 323 
of selection, 456, 468, 470-71 
of factor matrix, 92, 97-98, 269, 343 
of Gi’amian matrix, 82, 304 
in Heywood case, 289-90 
ideal, 317-18 

of matrix of primary factors, 412, 434, 
436 

of matnx of law scores, 323 
and minimum trace, 287-89 
and multiple solutions for eommunal- 
lties, 309 

as number of independent factors, 291- 
94 
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Rank —continued 

and order, 82, 282, 286, 484 
i eduction in, 204, 239-40, 269 
of second-order domain, 411-14, 417-22 
of the side correlations, 283 
of Speaiman hierarchy, 273 
trial, 308 

unit, see Unit rank 
zero, 260 

Rank one, see Unit lank 
Rank Older, 64, 367-69 
Reference abilities 

correlation between, 78 
defined, 69 

in factorial formulation, 68-76, 78 
notation for, 85 
oblique, 69 
orthogonal, 69 
postulation of, 51-59, 62-67 
relation of, to factor loadings, 73 
Refeience axes (sec also Refeience fiamc, 
Reference vectois) 
defined, 195 

determined by scoie surfaces, 321-22 
independence of, 90-91 
location of 

with respect to centroid, 151-52 
by unitary factois, 344 
oblique, 135-39, 194-205, 215, 323-24 
oithogonal, 38-43, 89, 195, 292, 327 
in pi incipal-axes solution, 473-510 
lotation of, see Rotation of axes 
Refeience factois, 71-73, 85, 99 
Reference frame ( see also Refeience axes, 
Reference vectois) 
arbitrary, 93-06, 119-24, 134, 327 
choice of, 177-79, 284-85 
and cori elated factors, 139-40 
criteria foi location of 
analytical, 356, 377-78 
giaphical, 108, 331-35 
statistical, 177-81 

determined by centroid method, 327, 
361-62 
effect on 

of factoring method, 363 
of sign reversals in P-mntrix, 96 
indeterminacy of location of, 473-74 
insertion of, into configuration, 92, 98, 
150-53 

location of, vn, 176-81, 321, 334-35, 
377-78 

oblique, vi-ix, 43, 135-39, 215, 323-24, 
421 

orthogonal, 38-43, 137, 139, 251, 327 
in puncipal-axes solution, 473-510 
i elation to 

of correlation matux, 92 
of factoi matnx, 78, 91-92 
lotation of, see Rotation of axes 
m simple structure, 327-28, 334-35, 377 
Reference traits, 54-55, 139 


Reference vectois (see also Reference axes 
Reference flame) 
adjustment of, 196 
bipolar, 396-98 
centioid, 327 
choice of axes as, 323-21 
cosines nf, 227 

dnection cosines of, 132-38, 193-06 
227-28, 251, 347, 349-51 
long, 196-205, 227-29, 241 
positive, 396-98 
primary, 396-409, 422 
i elation of, to hypei planes and pnmaty 
vectois, 193, 347—49 
residual, 308 
tual, 378-85, 390-401 
unit, 87, 09, 135, 195-96, 381-85 
Reflection of test vectors, 96-97, 149, 153- 
57, 165-60 
Regression equations 

m iterative procedure, 409-500 
foi piediction problems, 59-62, 68, 179, 
504, 511-16 
foi ? on n, 490-500 
foi s on x, 515-16 
for slope of regression line, 499 
weights in, 59-61, 511-16 
for a on s, 511-15 

Reliability 

coefficient, 83-86 
as diagonal entry, 83, 484 
and euor vauance, 84 
notation for, 86-80 
peifecl, 474 

l elation to oommunality, 83-85, 282, 
287 

of test, 83-80 
Residuals 

centroid-factor, 153-57, 163-65, 178 
in diagonal method of factoi mg. 111 
effoet of leflection on, 153-57 
genei al-factoi, 473 

geometucal mterpi etation of, 155-57 
minimizing of, 60-61, 82, 149, 177-78, 
295 

pnncipal-factoi, 177-79, 473-74, 481- 
85, 493-07 
zeio, 295 

Reversals, sign, 96-97, 149, 153—57, 165- 
66 

RobmBon, G , 45 
Roots 

of characloiistic equation, 44-45, 477- 
79, 483-84, 487-91, 494-97, 500-503 
of polynomial, 44-45 
Rotation of axes 

adjustment of pianos m, 241-51 
chock on lineal independence in, 239-40 
computational founulne for thiee lotft- 
tions, 205 

computational piocedure for, 194—216 
corrections in, 208-12, 248 
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and entiles of factor matnx, ix, xi, 181- 
82, 292 

invariance of scalar product under, 91, 
09 

in method of extended vectors, 241-58, 
377, 406 
methods of 

double-gioup, 408-10 
minimizing weighted sums, 396-408 
by plotting on spheie, 131-39 
single-plane, 216-24 
successive appioximation, 378-96 
by thiee-dimensional sections, 195, 
225-39, 377 

by two-dimensional sections, 194-95, 
224, 377 

oblique, 43, 131-39, 194-224 
orthogonal, 38-43, 108-10, 202 
from orthogonal lefeience fiame to prin¬ 
cipal axes, viii, 286, 474-77 
purpose of, 101, 197, 292, 325-26 
to simple structure, 181-82, 225, 258, 
343, 508 

size of cosines and choice of, 241 
in three dimensions, 131-39, 194-224 
m two dimensions, 108-10 
use of principal-factor matrix m, 509 
Rowveetoi, 14 


S-matux 

in lotation of axes, 197-205, 227-29, 241 
of scores, 08, 71, 77, 86, 93, 504-5 
Sampling errois, x, 260-61, 264, 267-68, 
283, 361 

Sampling thcoiy, xn, 269 
Saturations, see Factor loadings, Projec¬ 
tions of test vectors 


Scalar, 23 

Scalar product of test vectors 
corielations as, 89-91, 449 
defined, 37 
notation for, 380 
theorems on, 91, 94 
Scale 

of direction cosines, 137 
normalized, 367-69 
Science 

fundamental principle of, 291 
ideal constructs m, 51-55 
nature of, 51-55, 332 
of psychology, 52, 338 
unique solution in, 332 
Seine functions, 319—27, 365, 368-72 
Score matrix 
element of, 68 

factonal analysis of, 68-71, 504-5 
notation for, 71, 86 
relation of 

to correlation matrix, 77, 93 
to factor matrix, 71 
to population matnx, 71 
anspose of, 77 


Score smfaces, 321-24 
Scores 

composite, 327 

condensation of, 177-79, 503-8 
defined, 62, 73 
derived, 63 
deviation of, 323 

functions for, 68-69, 319-27, 365, 368- 
72 

as lineal combinations of scores on cor¬ 
related pnmary factors, 356-58 
normalized, 64-66, 367-69, 448 
prediction of 

factor scores from test, 511-15 
test scoics fiom factoi, 515-16 
processes underlying test, 503 
law, 63-67, 323, 367-69 
reproduction of original, 284 
selected foi factorial study, 319 
standard; see Standard scores 
standard deviation of, 323, 368, 443-50 
summation, 442-43 
ti ansformation of, 307-76 
true factor, 511 
true test, 62 
variance of, 86 

Scoring formula, 367 
Second-order factors 
defined, 411 

diagrammatic representations of, 416- 
20 

domain of, 411-14, 420-22 
factoi patterns foi, 418-19 
geneial, 421-34 

mterpi etation of, 416, 434, 436 
notation foi, 411-14 
number of hneaily independent, 414 
occur! ence of, vu-vin, 411 
parameters of, 411—16, 434—39 
physical example of 

four-dimensional, 427-34 
three-dimensional, 415—16 
relation to first-order factors, 411-39 
and tetrads, 272 
theones of, 439 
transformation for, 421-27 

Sectioning of matrix, 301-4 
Sections 

three-dimensional, 195, 225-39 
two-dimensional, 194-224, 377 

Selection 

complete, 470 

dependence of test scores and, 441-43 
effect of 

on communalities, 450—53, 455 
on correlations, 444-50, 452-55 
on dispersions, 443-47, 450, 454-55 
on factorial results, 55, 326, 332-33, 
360-61, 412-13, 438, 440-72 
on primary factors of, 456-64 
on simple structure of, 456-64 
geometrical representation of '’44-45 
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Selection— continued. 

multivariate, 443-44, 464-72 
partial, 471 
total, 471 

Otis-Kelley formula on restncted, 449- 
50 

partial, 470 

restricted, 449-50 

tests, 443-53, 458, 464, 470 

total, 470-71 

types of, 440 

univariate 

complete, 443, 456, 459—64, 470 
paitial, 443, 459-64 
vauable, 443-53, 458, 464, 470 
Self-con elation of Lest, 96-97, 352 
Sign-changmg methods, 96-97, 149, 153- 
57, 165-66 

Simple configuration, 58, 328, see also Sim¬ 
ple structure 

Simple order, 320-21, 328-29 
Simple structure 
alternative, 329-31 
best-fitting, 355 
bipolar, 189-93 
complete, 361, 363, 378 
concept of, v, 58, 181-82, 319-46 
configuiational types of, 184-93, 507 
criteria foi, 334-40, 355-50 
defined, 181, 328 
determuiacy of, 361, 303 
effect on 

of selection, 440 
of tieatment of scores, 369 
equation of oblique, 354-55 
geometucal interpietation of, 327-29 
graphical indications of, 247, 256-58, 
335, 349 

hypei planes of, 327-28, 331, 335 
incomplete, 216, 363, 378 
interpretation of, 324 
and invariance, 360-67, 456-64, 409-71 
isolation of, 181-82, 327-34, 343 
locating reference frame for 
analytical, 356, 377-78 
giaphical, 247, 257-58, 334-36, 340, 
349 

oblique, 328, 354-56 
orthogonal, 247, 328-29, 352 
overdetermination of, 182, 216, 329, 361, 
363-64 
positive, 342 

and positive manifold, 341-44 
prmeipal-axes solution veisus, 503, 508- 
10 

puipoBC of, 332—33 
in ) dimensions, 327-29, 347 
rotation to, 181-82, 225-58, 343, 508 
subspacee in, 327-28, 347, 377 
uniqueness of, 329-40 
vector model of, 126 

Simplified centroid method of factonng, 
159 


Simplified Multiple Factor Method bvl, T 
Thurstone, 159 

Simultaneous equations, 13-18, 291 476 
Single-common factor, 259-81, 284, 487- 

Single-factoi formula of Spearman. 273- 
76, 279-81, 284, 305 
Single-factor method 
ambiguity in, 268 

of dnect summation method, 277-78 
limitations of, for multiple-factor anal¬ 
ysis, 279-81 

loganthmic solution for, 277 
single-factor formula m, 273-76, 284 
305-6 

of Spearman, v-ix, 259-68, 272-76 
of successive approximations, 278-79 
tetrad differences proceduie as, 265-68 
Single-plane method of rotation, 216-24 
Sisam, C H , 1 
Skewness, 64, 367 
Slope of a line, 33, 499-500 
Snyder, V , 1 

Solid Analytical Oeometry and Deleimm- 
ants, by Arnold Dresden, 1 
Space 

augmented, 421-22 

common-factor, see Common-factor 
space 

euoi-faetor, 73-76, 84-86, 140 
total factor, 176 
total test, 08, 176, 324, 420 
unique-factor, 73-76, 79, 86, 177 
Space factor, 145, 324-26, 515 
Speaiman, Cbailes, v-ix, 188, 259-76, 289- 
93, 305-6, 421, 439, 473 
Specific-factor variance, via, 75, 84-85, see 
also Specificity 
Specific factors 
absence of, 84 

change from common to, 361 
conti ibution of, 79 
defined, 73 
sepai ation of, 287 
of Speaiman, 272-73 
in total vai rnnee, 73-76 
Specificity 
defined, 75 
estimation of, 84-85 
m factoi matrix of unit lank, 268 
notation and tennmology foi, 85 
sepauition from oiror vauance of, 84, 
287 

Hphoiieal model, 125-43 

Spliencal triangle, 130, 134, 147-48, 344, 
304 

Square root, calculation of, 45-46 
Squaring process in principal-axes solu¬ 
tion, 484^97 



INDEX 


533 


Standard deviation of distribution, 323, 
368, 443 50, 454-55, 465-66, 470 
Standard eiror, 261, 267-68 
Standard scores 
in abilities, 63, 70, 511-12 
in correlated primary factors, 356-58 
distribution of, 64-66, 368 
linear form of, 68-69 
matrix of, 68, 71 
normalized, 64, 368, 448 
notation foi, 85 
l eduction to, 64-66, 368 
suifaces foi, 323-24 

m tests, estimated from standard scoie 
in ability, 515-16 
m unitary factors, 343 


Statistic, 414 

Statistical Confluence Analysts by Means of 
Complete Regression Systems, by Rag- 
nar Fusch, 614 

Slatistioal RSsumS of the Spearman Two- 
Factor Theory, by Karl J. Holzinger, 
267 


Stephenson, William, 271 
Street Gestalt Completion teat, 333 
Stretching factoi, 64, 132, 324, 381, 448, 
455, 484 

Structuie, faetoual 
compelling, 335, 378 
defined, 87, 17G, 181, 327-28 
oblique, 347-59 
simple, see Simple stiucture 
Subjects of experimental population, 324- 
26, 360-61, 440, 470-72 
Subspaces, 327-28, 347, 349, 377 
Successive apptoximation method of lota- 
tion, 378-96 

Successive appioximation method foi unit 
rank, 278-79 

Summational pioccduies in 

centroid method, see Centroid method of 
factoring 

dcteimination of single-common factor, 
277-78 

estimation of communalities, 304-5 
grouping method of factoring, 111-16, 
149 


Surfaces, scoie, 321—24 
Syllogism test, 326 
Synonyms tests, 338, 508 


Temperamental tiaits, 322-24 
Test battones 

as exploiatoiy domains, 57—58, 68—70, 
325, 333, 506-8 

factorial invariance in, 332-33, 361-67, 
373-76 

indeterminacy in, 378 
Test coeliicients, 68-69, 71-73, 86 


Test score; see Scores 
Test vectors 

angular separations of, 93, 344-45 
centroid of, 151-53 
configuration of, *>ee Configuration 
constellations of, 184-85, 297-99, 301, 
314, 344—45 
defined, 87 
equation for, 148 
extended, 225-58 
geometrical models 
three-dimensional, 125-48 
two-dimensional, 87-100 
length of, 91, 99 
as linear combinations of 
primary vectors, 127-30, 352-53 
reference vectors, 147-48 
normalized, 131, 379, 401, 448 
null, 175, 181, 336, 459 
plotting of normalized, 131-39, 194-95 
in positive region, 342 
projections of, see Projections of test 
vectors 

properties of, 87-97 
reflection of, 153-57 
scalar product of, 89-91, 99 
in second-order domain, 422 
in simple structure, 181, 338-39 
unit, 91, 94, 127-28, 131 
Tests 

complexity of, 320, 326, 328, 333-35, 
351 

composite, 69, 268, 327, 514-15 
correlation between, 57, 63, 76-82, 89- 
91, 95 

correlation with primary factors of, 327 
defined, 62 

effect of selection of, 364, 440 
faetoual composition of 

invariance of, 325-26, 333, 360-63, 
372-75 

prediction of, 333, 366, 506-8 
geometrical interpretation of, 87, 99 
notation for, 86 
number, 73, 145, 333, 361-62 
number of, to determine common fac¬ 
tors, 82, 97-98, 262-65, 291-94, 307- 
11, 440 
parallel, 83 

parameters of, 325, 332-33 
personality, 322-24 
primary traits in terms of, 348 
jisychological nature of, 326 
pure. 351-52 

reliability of, 83-86, 286, 474 
with saturations in several factors, 326 
327 

scores on, see Scores 

selection, 443 

space, 145, 324-26, 361-62 

in tetrad difference equation, 266 

use of, 325-27 

validity coefficients of, 327, 516 
variance of, see Variance 
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Teats —continued 

verbal, 145, 333, 338, 361-62 
•weights of primary mental abilities for, 
69, 361 

Tetrad differences procedure 

characteristics of, 265-68, 274, 293, 439 
diagonal elements m, 266, 284 
equation for, 265-66, 273, 293 
matrix interpretation of, 266, 270 
Tcferflids 

evaluation of, 268, 270 
graphical analysis of, 270-72 
number of, 265-70 

relation of, to second-order minors, vi, 
266-67 

of subgroup in correlation matrix, 271 
vanishing, 266-68, 270-71 
Them y of Multiple Factors, by L. L Thur- 
stone, 175, 273, 277, 282, 473 
Theory of single common factor, 272 
Theory of two factors, 271-73 
Thomson, Godfrey H , ix, 269-70, 283, 
287-91, 443-66, 470 
Time limits, 367 
Trace of 

correlation matnx, 287-89 
plane, 197, 206-10, 226-29, 232 ff 
Trait configuration, 327-28, 334-35, 344- 
45 

Trait vectors; see Test vectors 
Traits {see also Tests) 
clusters of, 126-27 
configuration of, 327-28, 334-35 
correlation between, 146, 322 
defined, 62 
description of, 54-55 
descriptive categories as, 328-29 
genetic basis of mental, 333 
geometrical interpretation of primary, 
349-50 

personality, 322-24 
primary, 347-52 
reference, 54-55, 139 
temperamental, 322 
Transformation 
determinant of, 40 
direction cosines for, 38-43 
foi group factors, 424-26 
infinitesimal orthogonal, 343 
linear, 38, 301, 368 
matrix of, 38, 43, 134-39, 347 
monotonic, 369 

oblique, 43, 131-39, 194-224, 347-52, 
422-26 

orthogonal, 38-43, 109-10, 136-37, 202, 
231, 343, 352 424-25 
from orthogonal refoience frame to prin¬ 
cipal axes, vm, 286, 474-77 
for second-ordei factors, 422-27 
of test scores, 368-69 


Trapezoid population, 427-34 
Triangular configuration, 184-87, 192 
Triplet, 182 
Tryon, Robert, 344-45 
Tucker, Lodyard R , xm, 46, 354, 356, 358 
441, 459, 497, 500 
Turnbull, H W , 1, 512 
Two-factor theory, 267, 271-73 

Unique factors 

and common factois, 73-76, 272-73 
contribution of, to diagonal cells, 79 
defined, 182 
notation for, 86 
postulation of, 73-76 
and unity in diagonals, 284, 286 
Unique vanance 
absence of, 177 
determination of, 286-87 
effect of diagonal values on, 282-87 
eiroi vanance in, 74, 84 
experimental dependence and, 441-42, 
468 

m Heywood case, 289-90 
sepaiation of common-factor vanance 
and, 282 

specific variance in, 74, 84-85 
Uniqueness 

and centroid method, 161 
of communnlities, 307-11, 317-18 
of configuration, 92-93, 96, 178-80, 293 
coi rection for, 146-47, 208 
of factor solution, 93, 178, 180 
of paiametcrs, 329-31 
of pnmary factors, 332-34 
of principal-axes solution, 178, 474, 503 
of simple structuie, 329-31, 334-40 
of test, see also Unique variance 
complement of, 282 
defined, 75 
notation for, 85 
Unit of measurement, 367-68 
Unit rank 
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correction, 379-82 

extended, 225-58 

linear dependence of, 48-50 

in a matrix, 14 

models of, 125-28 

notation for, 233, 280 


primary; see Primary vectois 
primary trait, 348-54 
reference, see Reference vectors 
row, 14 

scalar product of two, 37, 89 
test, 87, see also Test vectors 
trait, see Test vectors 
tnal, 378-85 
volume, 415 

Vectors of Mind, The, by L I, Thurstone, 
v-vi, 283, 327, 344, 360-61 
Verbal factor, 145, 333, 338, 361-62, 508, 
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